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EXECUTIVE  SUMMARY 


This  Test  and  Evaluation  Master  Plan  (TEMP)  was  approved  by  the 
Test  Policy  and  Review  Committee  (TPRC)  on  July  21 ,  1995,  in 
preparation  for  the  system  limited  production  phase  of  Key 
Decision  Point  3  (KDP-3).  This  document  addresses  the  testing 
requirements  for  the  Initial  Operational  Capability  (IOC)  of  the 
Integrated  Terminal  Weather  System  (ITWS)  program.  An  updated 
TEMP  will  be  developed  with  additional  detail  as  the  program 
progresses  through  KDP-4,  to  ensure  compliance  with  program 
objectives.  Subsequent  versions  of  this  TEMP  will  be  submitted 
for  approval  by  the  TPRC.  The  results  of  both  Development  Test 
and  Evaluation  (DT&E)  and  Operational  Test  and  Evaluation  (OT&E) 
testing  will  be  utilized  as  input  for  a  deployment  recommendation 
decision . 

This  ITWS  TEMP  lays  the  foundation  for  the  ITWS  test  strategy, 
resources,  implementation  and  organization  responsibilities.  The 
test  efforts  governed  by  this  TEMP  will  ensure  ITWS  meets  the 
system  and  subsystem  requirements  allocated  to  the  project  as 
defined  by  the  NAS-SS-1000,  NAS-SR-1000,  Operational  Requirements 
Document  (ORD)  and  FAA-E-2900  (ITWS  System  Specification).  This 
TEMP  further  describes  the  test  and  evaluation  components  for 
meeting  program  objectives  for  each  acquisition  phase.  ITWS  will 
follow  the  procedures  for  OT&E  stated  in  Federal  Aviation 
Administration  (FAA)  Order  1810.4B.  The  ITWS  program  has  been 
designated  for  ATQ,  Independent  Operational  Test  and  Evaluation 
(lOT&E)  Oversight.  The  TEMP  format  is  in  accordance  with  FAA- 
STD-024b. 

The  ITWS  procurement  will  meet  Mission  Need  Statement  (MNS) 
requirements  by  using  Commercial-Off -The-Shelf  (COTS)  hardware 
and  Government  Furnished  Equipment  (GFE)  algorithms  for  ITWS 
product  generation.  The  ITWS  program  completed  the  Demonstration 
Phase  as  defined  by  FAA  Order  1810. IF,  and  is  into  KDP-3.  The 
development  and  approval  of  this  TEMP  supports  the  KDP-3  decision 
to  progress  to  the  next  phase. 

The  ITWS  integrates  weather  data  from  terminal  area  sensors  to 
provide  value-added,  real-time  products  useable  without 
meteorological  interpretation.  ITWS  products  will  be  tailored 
for  immediate  use  by  terminal  air  traffic  controllers,  traffic 
managers,  and  automated  traffic  management  systems.  There  are 
34  systems  planned  for  deployment. 

Demonstration/Validation  (DEMVAL)  OT&E  took  place  in  Dallas  Fort 
Worth  and  Orlando  in  1993  and  in  Memphis  and  Orlando  in  1994. 

OT&E  Integration  and  OT&E  testing  is  planned  to  begin  in  the  1998 
timeframe  and  will  occur  at  the  FAA  Technical  Center,  simple  site 
airport  (e.g.,  single  sensors)  and  complex  site  airport  (e.g., 
multiple  sensors).  OT&E  is  planned  to  be  completed  during  the 
1st  quarter  1999. 


1.  INTRODUCTION. 


1 . 1  BACKGROUND . 

This  Integrated  Terminal  Weather  System  (ITWS)  Test  and 
Evaluation  Master  Plan  (TEMP)  lays  the  foundation  for  the  ITWS 
test  strategy,  resources,  and  implementation  responsibilities. 

The  test  efforts  governed  by  this  TEMP  will  ensure  the  ITWS  meets 
the  system  and  subsystem  requirements  allocated  to  the  project  in 
the  NAS-SS-1000,  NAS-SR-1000,  and  FAA-E-ITWS  (ITWS  System 

Specification) .  This  TEMP  further  describes  the  Test  and 
Evaluation  (T&E)  components  for  meeting  program  objectives  for 
each  acquisition  phase.  The  ITWS  will  follow  the  procedures  for 
Operational  Test  and  Evaluation  (OT&E)  stated  in  FAA  Order 
1810.4B.  The  TEMP  format  is  in  accordance  with  FAA-STD-024b . 

The  ITWS  integrates  weather  data  from  terminal  area  sensors  to 
provide  value-added,  real-time  products  that  need  no 
meteorological  interpretation.  ITWS  products  will  be  tailored 
for  immediate  use  by  terminal  air  traffic  controllers,  traffic 
managers,  and  automated  traffic  management  systems.  There  are  37 
systems  planned  for  development  (34  operational/3  test/training). 

The  ITWS  procurement  will  meet  Mission  Need  Statement  (MNS) 
requirements  by  using  Commercial-Off -The-Shelf  (COTS)  hardware 
and  Government  Furnished  Equipment  (GFE)  algorithms  for  ITWS 
product  generation.  Additionally,  the  software  used  by  the 
Massachusetts  Institute  of  Technology  Lincoln  Laboratory  (MIT/LL) 
to  implement  the  algorithms  will  be  supplied  to  the  contractor  as 
Government  Furnished  Information  (GFI).  The  ITWS  program  has 
received  conditional  KDP-3  approval.  Final  approval  is 
contingent  on  the  ITWS  TEMP  being  approved. 

This  TEMP  addresses  the  testing  requirements  for  the  Initial 
Operational  Capability  (IOC)  of  the  ITWS  program.  An  updated 
TEMP  will  be  developed  with  additional  detail  as  the  program 
progresses,  through  KDP-3  and  KDP-4,  to  ensure  compliance  with 
program  objectives.  This  TEMP  and  subsequent  versions  will  be 
submitted  for  approval  by  the  Test  Policy  Review  Committee 
(TPRC).  The  results  of  both  Development  Test  and  Evaluation 
(DT&E)  and  OT&E  testing  will  be  utilized  as  input  for  a 
deployment  recommendation  decision. 

Demonstration/Validation  (DEMVAL)  OT&E  took  place  in  Dallas/Fort 
Worth  (DFW)  and  Orlando  (MCO)  in  1993,  and  in  Memphis  (MEM)  and 
MCO  in  1994.  Additional  evaluations  in  the  operational 
environment  are  scheduled  at  the  DFW  metroplex  for  the  summer  of 
1995.  These  DEMVALs  mitigated  risk  by  allowing  the  user  and  test 
community  to  evaluate  ITWS  product  suitability,  usefulness,  and 
meteorological  validity  in  an  operational  environment  and 
determined  the  feasibility  to  proceed  to  full-scale  development. 
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The  results  of  the  demonstration  phase  OT&E  verified  that  the 
weather  products  are  acceptable  to  the  user  community = 

Real-time  source  weather  information,  interfaces  and  test  data 
sets  generated  by  a  contractor  built  Test  Tool,  are  planned  to  be 
used  as  input  for  weather  scenarios  at  first  article  sites.  OT&E 
testing  is  planned  to  begin  in  the  1998  timeframe  at  the  FAA 
Technical  Center  and  at  simple  (e.g.,  single  airport  and  single 
weather  radar  inputs)  and  complex  (e.g.,  multiple  airports  and 
multiple  radar  inputs)  airports.  OT&E  is  planned  to  be  completed 
during  the  1st  quarter  1999. 

The  ITWS  project  (CIP  63-21)  is  a  Level  I  major  system 
acquisition  which  will  provide  coverage  for  the  45  airports 
having  Terminal  Doppler  Weather  Radar  (TDWR) .  Some  of  these 
airports  are  supported  from  one  common  Terminal  Radar  Approach 
Control  (TRACON)  facility  or  a  Metroplex  Control  Facility  (MCF) ; 
therefore,  only  34  operational  ITWS  processing  elements  will  be 
required.  The  ITWS  program  has  been  designated  for  oversight  by 
the  Office  of  Independent  Operational  Test  &  Evaluation  (lOT&E) 
per  FAA  Order  1810. IF. 

The  National  Airspace  System  Change  Proposal  (NCP)  17331  dated 
April  13,  1995,  and  the  Operational  Requirements  Document  (ORD), 
dated  February  1995  were  the  primary  source  documents  used  for 
the  development  of  this  TEMP. 

1 . 2  PURPOSE . 

The  purpose  of  this  TEMP  is  to  define  the  overall  T&E  strategy 
necessary  to  ensure  the  successful  integration  of  the  ITWS  into 
the  National  Airspace  System  (NAS)  and  to  assure  the  operational 
suitability  and  effectiveness  of  the  ITWS.  This  TEMP  describes 
the  T&E  processes  that  will  be  used  to  ensure  the  ITWS  meets  the 
user  and  system  requirements  and  is  operationally  ready. 

The  ITWS  TEMP  test  strategy  includes;  (1)  defining  test 
methodology;  (2)  verifying  requirements,  Critical  Operational 
Issues  (COI),  Critical  Performance  Parameters  (CPP),  and  Minimum 
Acceptable  Operational  Performance  Requirements  (MAOPR) ,  and  (3) 
identifying  organizational  roles  and  responsibilities.  ITWS 
COIs,  CPPs,  and  MAOPRs  are  defined  in  the  ITWS  ORD.  This  TEMP  is 
developed  in  accordance  with  FAA  Order  1810. 4B  and  FAA-STD-024b . 

A  Verification  Requirements  Traceability  Matrix  (VRTM)  containing 
high-level  functional  and  performance  requirements  to  be  tested 
during  the  ITWS  T&E  program  is  included  in  appendix  A. 

1.3  SCOPE. 

This  ITWS  TEMP  specifically  addresses  the  IOC  ITWS  which  will  be 
tested  during  the  Development  Phase  and  subsequent  Production 
Phase.  The  FAA  T&E  overview  begins  with  the  MNS  and  continues 
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through  Production  Acceptance  Test  and  Evaluation  (PAT&E).  The 
testing  that  has  taken  place  during  the  Demonstration  Phase  and 
the  testing  that  will  be  performed  during  the  Development  Phase 
will  ensure  that  the  IOC  ITWS  satisfies  KDP-3  exit  criteria,  NAS- 
SS-1000  Specifications,  MAOPR,  and  CPP  requirements. 

The  FAA  Technical  Center  will  be  used  for  the  initial  phase  of 
OT&E  testing.  The  testing  done  on  the  system  at  the  FAA 
Technical  Center  will  assure  NAS  interfaces  and  operational 
functionality  before  the  ITWS  is  placed  in  an  operational  field 
site . 

OT&E  Operational  and  Shakedown  testing  will  be  conducted  at  two 
operational  field  sites.  The  selected  simple  and  complex  site 
will  offer  a  broad  spectrum  of  convective  activity  including  both 
air  mass  differentials  and  frontal  systems.  The  simple  site  will 
assess  the  minimum  interface  functionality,  while  the  complex 
site  will  assess  the  interfaces  with  multiple  airports  and  radar 
systems  within  an  operational  environment.  Both  sites  will 
assess  the  interface  functionality,  operational  effectiveness, 
and  suitability  within  an  air  traffic  environment.  In  addition, 
the  contractor's  implementation  of  the  algorithm  will  be  verified 
during  the  entire  test  cycle. 

The  KDP-2  ITWS  Acquisition  Memoranda  did  not  identify  any 
technical  capabilities  that  had  to  be  met  prior  to  entering  KDP- 
3.  In  addition,  since  the  operational  capabilities  demonstrated 
during  Demonstration  Phase  OT&E  are  not  commercially  available, 
there  is  no  value  to  performing  an  Operational  Capability 
Demonstration  (OCD) . 

2 .  REFERENCE  DOCUMENTS . 

The  following  specifications,  standards,  publications,  orders, 
and  other  miscellaneous  documents  were  used  in  preparation  of 
this  document  as  well  as  subsequent  lower  level  test  documents 
and  test  reports . 

FAA  DOCUMENTS 


FAA  Specifications 

NAS-SS-1000  NAS  System  Specification  Volume  I,  Functional  and 
Performance  Requirements  for  the  National  Airspace 
System  General . 

NAS-SS-1000  NAS  System  Specification  Volume  II,  Air  Traffic 
Control  Element  Requirements  for  the  National 
Airspace  System. 
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NAS-SS-1000 

NAS  System  Specification  Volume  III,  Functional 
and  Performance  Requirements  for  the  Ground-to-Air 
Element . 

NAS-SS-1000 

NAS  System  Specification  Volume  IV,  Functional  and 
Performance  Requirements  for  the  NAS 

Communications  Element. 

NAS-SS-1000 

NAS  System  Specification  Volume  V,  Functional  and 
Performance  Requirements  for  the  National  Airspace 
System  Maintenance  and  Operations  Support 

Elements . 

NAS-SR-1000 

NAS  System  Requirements  Specification. 

NCP  17331 

NAS  Change  Proposal,  April  13,  1995. 

FAA-E-ITWS 

Specification  for  the  Integrated  Terminal  Weather 
System,  March  1995. 

FAA  Standards 

FAA- STD-021 

Configuration  Management  [Contractor 

Requirements],  August  17,  1987. 

FAA-STD-024b 

Preparation  of  Test  and  Evaluation  Plans  and  Test 
Procedures,  August  22,  1994. 

FAA-STD-026 

NAS  Software  Development,  August  4,  1993. 

FAA-STD-039 

NAS  Open  Systems  Architecture  and  Protocols, 
October  28,  1991. 

FAA-STD-047 

National  Airspace  System  (NAS)  Open  Systems 
Interconnection  (OSI)  Conformance  Testing, 

December  1993. 

Other  FAA  Publications 

NAS-MD-793A 

Remote  Maintenance  Monitoring  System  Functional 
Requirements  for  the  Remote  Monitoring  Subsystem 
(RMS) . 

FAA  Order 

1800. 8F 

NAS  Configuration  Management,  May  20,  1991. 

FAA  Order 

1810. IF 

FAA  Acquisition  Process,  March  19,  1993. 

FAA  Order 

1810. 4B 

FAA  NAS  Test  and  Evaluation  Policy,  October  22, 

1992  . 
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ITWS  ORD 


Risk 

Management 


Integrated  Terminal  Weather  System  Operational 
Requirements  Document  (ORD),  February,  1995. 

Integrated  Terminal  Weather  System  (ITWS)  Risk 
Management  Plan  (Draft)  September  1994. 


Interface  Documents. 


NAS-IR-25082514 


NAS-IR-25142513 


NAS-IR-31052514 


NAS-IR-31052514 


NAS-IR-TBD 


NAS-IR-43020001C 


NAS-IR-43020001C 


NAS-IR-51035101 


NAS-IR-43034001 


Interface  Requirements  Document,  AWOS  Data 
Acquisition  System  to  the  Integrated  Terminal 
Weather  System  (ADAS/ITWS),  April  3,  1995. 

Interface  Requirements  Document,  Integrated 
Terminal  Weather  System  to  the  Data  Link 
Processor  ( ITWS/DLP2A) ,  April  3,  1995. 

Interface  Requirements  Document,  Integrated 
Terminal  Weather  System  to  Terminal  Doppler 
Weather  Radar  (TDWR/ITWS),  Part  One,  Draft, 
April  26,  1995. 

Interface  Requirements  Document,  Integrated 
Terminal  Weather  System  Situation  Display  to 
Terminal  Doppler  Weather  Radar  (TDWR 
SD/ITWS),  Part  Two,  Draft,  April  18,  1995. 

Interface  Requirements  Document,  Airport 
Surveillance  Radar  -  Model  9  (ASR-9)  Weather 
Channel  with  the  Integrated  Terminal  Weather 
System  (ITWS)  ASR-9  (Weather  Channel ) /ITWS , 
Draft,  February  15,  1994. 

National  Airspace  Data  Interchange  Network 
(NADIN)X.25  Packet  Mode  Users  Interface 
Requirements  Document,  March  1992.  (Appendix 
for  Users,  March  1995). 

National  Airspace  Data  Interchange  Network 
(NADIN)X.25  Packet  Mode  Users  Interface 
Requirements  Document,  March  1992.  (Appendix 
for  ACF  SD,  March  1995). 

Interface  Requirements  Document,  Remote 
Monitoring  Subsystem/Maintenance  Processor 
Subsystem  (RMS/MPS),  December  1994. 

Interface  Requirements  Document,  National 
Weather  System  to  NWSTG/NAS  Users  system, 
October  24,  1994. 


Unisys  12083041  Interface  Control  Document,  for  the 
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SOW  ITWS  Statement  of  Work 

ACT-320  Report  ITWS  Demonstration/Validation  Phase  OT&E 

Final  Report,  (DRAFT),  February  1995. 

DOT/FAA/CT-TN  95/1  Final  Report  for  the  Air  Traffic  Control 

(ATC)  Operational  Evaluation  of  the  Prototype 
Integrated  Terminal  Weather  System  (ITWS)  at 
Dallas/Fort  Worth  (DFW)  and  Orlando 
International  (MCO)  Airports  (May  -  September 
1993)  . 

DOT/FAA/RD-95/7  ITWS  Demonstration  and  Validation  Operational 

Test  and  Evaluation  Report,  April  13,  1995. 

DOT/FAA/ND-95/11  ITWS  Algorithm  Specification,  Volume  I  and 

II,  May  31  1995 

Human  Factors  Plan  Integrated  Terminal  Weather  System  (ITWS) 

Human  Factors  Plan,  March  1995. 

Project  Report  Integrated  Terminal  Weather  System  (ITWS) 

ATC-203  1992  Annual  Report  Lincoln 
Laboratory,  Massachusetts  Institute  of 
Technology,  September  7,  1993. 

3.  INTEGRATED  TERMINAL  WEATHER  SYSTEM  DESCRIPTION. 

3.1  ITWS  IMPLEMENTATION. 

The  ITWS  hardware  will  consist  of  a  COTS-based  computer  system 

and  associated  hardware  (e.g.,  modems,  racks,  etc.).  The  ITWS 

Algorithm  Specification  developed  by  MIT/LL  will  be  provided  to 

the  contractor  as  GFE.  The  software  code  that  MIT/LL  used  to 
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implGment  thG  algorithms  during  thG  DEMVAL  will  bG  providGd  to 
thG  contractor  as  GFI. 

3.2  ITWS  FUNCTIONAL  DESCRIPTION. 

ThG  ITWS  will  collGCt,  intGgrate,  and  procGSS  wGathor  data  from 
FAA  and  National  Waathor  SarvicG  (NWS)  sansors  and  from  aircraft 
in  thG  tarminal  araa  to  provida  valua-addad,  raal-tima  products 
that  ara  usabla  without  mataorological  intarpratation.  Thasa 
products  include  current  terminal  area  weather  and  short  term  (0~ 
30  minutes)  predictions  of  significant  weather  phenomena.  ITWS 
products  will  be  generated  for  immediate  use  and  are  available  to 
air  traffic  control  (ATC)  personnel,  traffic  managers, 
supervisors,  automated  traffic  management  systems,  and  pilots  via 
data  link.  ITWS  products  will  also  be  provided  to  other  users 
with  defined  requirements. 

Figure  3.2-1  is  a  high  level  presentation  of  the  ITWS  information 
flow.  Table  3.2-1  presents  the  information  in  greater  detail  for 
each  of  the  ITWS  products  and  their  associated  capabilities. 

The  ITWS  will  receive  TDWR,  ASR-9,  and  Next  Generation  Weather 
Radar  (NEXRAD)  radar  data.  The  ITWS  will  integrate  this  data 
with  gridded  data  and  Meteorological  Data  Collection  and 
Reporting  System  (MDCRS)  data  received  via  the  National  Weather 
System  Telecommunication  Gateway  (NWSTG) .  Automated  Weather 
Observing  System  (AWOS),  Automated  Surface  Observing  System 
(ASOS),  Automated  Lightning  Detection  and  Reporting  System 
(ALDARS)  are  received  from  the  ASOS/AWOS  Data  Acquisition  (ADAS). 
Data  input,  assimilation,  and  processing  for  product  generation 
will  occur  in  the  ITWS  product  generator  located  at  the  TRACON 
building.  The  ITWS  graphics  and  text  products  will  be  presented 
to  supervisors  and  traffic  managers  on  the  Situation  Display 
(SD).  ITWS  text  products  will  be  presented  to  air  traffic 
controllers  on  the  Ribbon  Display  Terminal  (RBDT) .  The  ITWS  is 
required  to; 

1.  Process  data  (real-time  weather  observation  data). 
Aviation  Impact  Variables  (AIV) ,  and  State  of 
Atmosphere  Variables  (SAV).  This  processing  includes 
performing  data  assimilation,  interpolation,  and 
running  extraction  and  tracking  algorithms. 

2 .  Extract  information  from  existing  Aviation  Weather 
Products  (AWP)  as  input  for  generating  new  AWPs . 

Provide  direct  or  pass-through  sensor  data.  This  data 
can  be  used  as  received  or  as  base  data  for  a  value 
added  product. 


3. 


TABLE  3.2-1.  ITWS  IOC  PRODUCTS 


rrws  PRODUCTS  capability 

a. 

Windshear: 

1)  Microburst 
deteclion/prediction 

2)  Gust  front  detection  and 
forecast 

3)  Ribbon  display  alerts 

4)  Microburst  alert  Automated 
Terminal  Information  System 
(ATIS)  timer 

5)  Wind  shear  alert  ATIS  timer 

6)  Gust  front  impact  timer 

Accurate  detection/prediction  and  alerting  of 
microbursts  including  location,  runway  impact 
and  intensity;  Improved  gust  front  detection  and 
forecasts;  Timers  (ATIS/700  and  Gust  front 
impact); 

b. 

Gust  front  wind  shift  estimate 

Estimate  of  wind  speed  and  direction  10 
minutes  behind  the  gust  front; 

C. 

Precipitation; 

1)  5  nautical  mile  range 

2)  TRACON  range 

3)  100  nautical  mile  range 

4)  200  nautical  mile  range 

Precipitation  intensity,  location  and  extent  in  4 
ranges;  TRACON  precipitation  with  ASR-9  AP 
removed; 

d. 

Storm  motion  and  extrapolated 
position: 

1)  5  nautical  mile  range 

2)  TRACON  range 

3)  100  nautical  mile  range 

4)  200  nautical  mile  range 

Indication  of  storm  speed  and  direction;  Near- 
term  projected  storm  location,  and  extent 
depicted  in  4  ranges; 

e. 

Storm  cell  information: 

1)  5  nautical  mile  range 

2)  TRACON  range 

3)  100  nautical  mile  range 

4)  200  nautical  mile  range 

Detailed  data,  on  request,  indicating  storm 
features  including;  hail,  lightning,  mesocyclone 
and  echo  tops  in  4  ranges; 

f. 

ASR-9  AP; 

1)  Precipitation  with  AP 
flagged 

2)  AP  alert 

indication  of  location  and  extent  of  AP  in  the 
ASR-9  reflectivity;  Alerting  to  the  presence 
and  location  of  ASR-9  AP; 
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g- 

Tornado: 

1)  Detection 

2)  Alert 

Indicate  locations  on  SD  in  4  ranges;  Alert  to 
the  presence  of  tornadoes  within  designated 
distances  of  each  ITWS  airport; 

■ 

Airport  lightning  warning 

Indication  of  lightning  within  designated 
distances  of  each  ITWS  airport; 

LLWAS  winds 

Centerfield  and  runway -specific  winds  as 
designated  to  cover  each  ITWS  airport; 

J- 

Terminal  winds: 

1)  Gridded  wind  field 

2)  Wind  profile 

Profiles  of  winds  for  each  ITWS  airport  for 
designated  reference  points -and  altitudes  for 
display; 

k. 

Runway  configuration 

Airport  configuration  (runway  configuration); 

1, 

Terminal  weather  text  message 

Provides  a  textual  weather  message  for  terminal 
area  defining  pressure,  weather,  visibility, 
ceiling,  winds  and  remarks. 

3.2.1  ITWS  Unique  Operational  and  Performance  Characteristics, 

ITWS  products  are  generated  from  data  received  from  various  input 
sensors  and  integrated  into  a  single  display  and  presented  to  end 
users  in  a  form  that  does  not  require  further  meteorological 
interpretation.  The  ITWS  will  archive  generated  products,  system 
status  messages,  and  user  inputs  for  a  15-day  cycle. 

Additionally,  the  ITWS  will  archive  acquired  data  used  for  ITWS 
product  generation  for  a  6-hour  period. 

3.3  INTERFACES. 

The  ITWS  will  interface  with  various  FAA  and  NWS  weather  systems 
to  fulfill  its  mission  requirements.  ITWS  interfaces  are  divided 
into  two  categories;  input  interfaces  and  output  interfaces 
(users)  as  shown  in  figure  3.3-1.  The  following  subsections 
describe  each  interface. 

Each  interface  will  achieve  interoperability  and  compatibility 
using  the  International  Standards  Organization  (ISO)  Open  System 
Interconnection  (OSI)  reference  model,  in  accordance  with  ISO 
7498  and  FAA-STD-047.  Communication  architecture  will  be 
designed  in  accordance  with  FAA-STD-039.  Messages  and  related 
format  requirements  are  presented  in  each  Interface  Requirements 
Documents  (IRD)  and  Interface  Control  Documents  (ICD). 
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3.3.1  External  Input  Interfaces. 


a . 

TDWR/ ITWS 

b. 

LLWAS/ITWS 

c , 

ASR-9/ITWS 

d . 

NEXRAD/ITWS 

e . 

NMC/ITWS 

f . 

ADAS /ITWS 

3. 3. 1.1 

TDWR/ ITWS. 

This  is  a  two-part  interface.  For  Part  I,  the  TDWR  will  provide 
the  ITWS  product  generator  with  data  formatted  and  transmitted  in 
accordance  with  NAS-IR-31052514 ,  Part  I.  The  data  provided  will 
include  TDWR  base  data  and  Low  Level  Windshear  Alert  System 
(LLWAS)  products. 

For  Part  II,  the  TDWR  will  provide  a  direct  interface  to  each 
ITWS  SD  in  the  TRACONs  and  Airport  Traffic  Control  Towers  (ATCT) . 
This  will  provide  a  backup  to  be  used  by  the  SD  and  accompanying 
RBDTs  in  case  of  failure  to  receive  data  from  the  ITWS  product 
generator.  Data  and  protocols  will  be  in  accordance  with  NAS-IC- 
31052514  Part  II. 

3. 3. 1.2  LLWAS/ITWS. 

The  ITWS  will  interface  to  either  LLWAS  II  or  LLWAS  III  at  each 
ITWS  airport  SD.  The  LLWAS  II  will  provide  threshold  and  center 
field  wind  data  exclusively.  The  LLWAS  III  will  supply  runway 
oriented  winds  and  windshear  products.  LLWAS  data  will  be 
provided  to  the  TDWR  Radar  Product  Generator  (RPG)  for 
distribution  to  the  ITWS  product  generator  and  for  integration 
and  distribution  to  the  ITWS  SD.  LLWAS  II/III  also  provides  data 
directly  to  the  SD  as  a  backup  to  a  TDWR  and/or  communications 
failure . 

3. 3. 1.3  ASR-9/ITWS. 

ASR-9  will  provide  digitized  6-level  weather  reflectivity  data  in 
accordance  with  the  ASR-9/ITWS  NAS-IR-TBD.  This  document  is 
currently  being  updated.  This  IRD  defines  the  weather  data  to  be 
provided  by  the  ASR-9.  The  ITWS  will  merge  all  ASR-9  inputs 
covering  the  TRACON  area. 
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ITWS  Informat  bn  I  H  Display 

User  I  1  Terminal 


3.3. 1.4  NEXRAD/ITWS. 

NEXRAD  will  provide  products  in  accordance  with  the 
RPG/associated  Principle  User  Processor  (PUP)  ICD  (Unisys 
12083041).  The  products  will  include  the  following  data:  storm 
structure,  storm  tracking,  echo  tops,  hail  index,  mesocyclone, 
tornado  vortex  signature,  mean  radial  velocity,  and  layered 
composite  reflectivity.  The  data  format  and  the  communication 
protocol  is  defined  in  the  NEXRAD/ITWS  ICD. 

3. 3. 1.5  NMC/ITWS. 

The  NWS  will  provide  gridded  data  (including  Rapid  Update  Cycle 
[RUC])  and  MDCRS-processed  airborne  observations  to  the  ITWS. 

The  information  will  be  broadcast  via  high  speed  communications 
links .  The  communication  protocols  and  subnetwork  communications 
are  defined  in  accordance  with  the  NWSTG/NAS  User  IRD  (NAS~IR~ 
43034001) . 

3. 3. 1.6  ADAS/ITWS. 

ADAS  will  provide  the  ITWS  with  automated  surface  observations 
and  lightning  network  information  via  the  NADIN  Packet  Switching 
Network  (PSN).  The  provided  ADAS  information  will  be  in 
accordance  with  the  NAS-IR~250825 14 .  ADAS  will  provide  that 
portion  of  the  lightning  information  which  falls  within  the 
geographical  area  of  the  Area  Control  Facility  (ACF) .  All  data 
exchange  at  the  application  layer  is  in  the  World  Meteorological 
Organization  (WMO)  format. 

3.3.2  External  Output  Interfaces. 

The  set  of  products  disseminated  from  ITWS  to  output  systems  will 
be  tailored  to  meet  the  requirements  of  each  interface/user. 

a.  ITWS/External  Users, 

b.  ITWS/Data  Link  Processor  (DLP), 

c.  ITWS  RMS/Maintenance  Processor  Subsystem  (MPS), 

d.  ITWS/National  Airspace  Data  Interchange  Network  (NADIN). 


3.3.2. 1  ITWS/External  Users. 

Output  ports  will  be  provided  to  transmit  terminal  weather 
products  for  access  by  external  users  such  as  airlines  and  the 
NWS.  The  format  of  messages  provided  over  these  ports  will  be 
consistent  with  those  defined  in  the  ITWS  product  generator  to 
ITWS  SD  interface  ICD. 
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3. 3. 2. 2  ITWS/DLP2A. 


ITWS  products  will  be  provided  to  the  DLP  for  dissemination  to 
pilots  in  accordance  with  NAS-IR-25142513 .  This  interface  is 
between  the  ITWS  and  the  DLP  located  in  each  Area  Control 
Facility  (ACF) »  The  application  processes  will  provide  only 
those  services  required  for  the  transfer  of  ITWS  products  to  the 
DLP  (one-way  operation).  Interface  functional  requirements  will 
be  in  accordance  with  ISO  7498. 

3. 3. 2. 3  ITWS  Remote  Monitoring  System  (RMS)/  Maintenance 
Processor  Subsystem  (MPS) . 

Each  ITWS  will  incorporate  an  RMS,  which  will  supply  system 
status  to  the  MPS.  The  interface  will  be  in  accordance  with  NAS 
IR-51035101.  The  NAS-MD-793a  provides  the  RMS  design 
implementation  requirements  and  will  be  used  in  conjunction  with 
this  IRD.  The  MPS  and  RMS  application  processes  will  exchange 
messages  that  the  Remote  Monitoring  Subsystem  (RMS)  is  required 
to  provide  to  the  MPS . 

3. 3. 2. 4  ITWS/National  Airspace  Data  Interface  Network  II _ ( NAD Ijj 

II)  Packet  Switching  Network  (PSN)  User. 

NADIN  will  be  used  as  the  communications  medium  between  ITWS  and 
the  external  systems  to  provide  the  data  for  the  exchange  of 
messages.  The  requirements  for  the  connectivity  are  defined  in 
NAS-IR-43020001C . 

3.4  CRITICAL  PERFORMANCE  PARAMETERS  (CPP). 

The  ITWS  CPPs  are  presented  in  table  3.4-1.  The  objectives  and 
thresholds  are  extracted  from  the  ITWS  ORD.  The  thresholds  are 
baselined  in  NAS-SS-1000  as  required  NAS  parameters.  A 
performance  parameter's  threshold  is  the  minimum  value  necessary 
to  provide  an  operational  capability  that  will  satisfy  the 
mission  need.  Performance  objectives  are  defined  as  values 
beyond  the  threshold  that  should  reflect  an  operationally 
meaningful,  measurable,  and  affordable  improvement  on  operations 
or  support  beyond  that  provided  by  the  threshold  value. 


TABLE  3.4-1. 


ITWS  CRITICAL  PERFORMANCE  PARAMETERS 


CRITICAL 

PARAMETER 

THRESHOLD 

OBJECTIVE 

A)  General  Performan 

ce 

1)  End-to-end 
availability 

Essential  Service  for  wind  shear 
(MB,  WS,  &  GF)  requirements 
(end-to-end  availability  >/=.999) 

Objective  =  Threshold 

2)  ITWS  allocated 
availability 

>/=.99981 

Objective  =  Threshold 

3)  ITWS  reliability 

>/=2704  hours  mean  time 
between  failure  (MTBF) 

Objective  =  Threshold 

4)  Coverage  area 

Surface-23,000  feet  above 
ground  level  (AGL)  out  to  30 
nm  beyond  the  TRACON 
boundary,  product  dependent 

Objective  =  Threshold 

5)  Data  retention 

6  hours  of  input  data 

Objective  =  Threshold 

6)  Product 
archiving 

15  days  of  ITWS  products 
directly  supporting  display  or 
user  output 

Objective  =  Threshold 

7)  Automatic 
recovery  on 

ITWS  generation 
failure 

Switch  to  TDWR  display  within 

30  seconds  of  ITWS  wind  shear 
product  outage 

Objective  =  Threshold 

8)  Timeliness  of 
reporting  weath¬ 
er  phenomenon 
(tornado,  hail, 
mesocycione) 

</=l  minute  of  receipt  of 
applicable  data 

Objective  =  Threshold 
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B)  Wamid  sheair  asud  stomnm  piradyicfe  geimeirsiItBOini  pesfomBSiinice 


1)  Microburst  Prediction 


a)  Probability  of 
false  microburst 

alert 

</=0.1 

</=0.05 

b)  Prediction  lead 
time 

<1-1  minutes,  prior  to  onset  of 
microburst  for  60%  of  predicted 
valid  wet  microbursts 

</=2  minutes,  prior  to  onset  of 
divergent  wind  shear  for  90%  of 
wet  predicted  events 

2)  Gust  Front  Forecast 

a)  Predicted 
position  time 

Position  predicted  10  minutes 
and  20  minutes  in  advance 

Objective  =  Threshold 

b)  Predicted  posi¬ 
tion  accuracy 

Predict  70%  of  gust  fronts 
impacting  airport  with  wind 
change  >/=15  knots  10  minutes 
in  advance 

Predict  90%  of  gust  fronts 
impacting  airport  with  wind 
change  >/=15  knots  10  minutes 
m  advance 

c)  Probability  of 
false  prediction 

Probability  of  false  10  minute 
prediction  </=  O.IO  for  gust 
fronts  with  wind  change>/=15 
kts 

Probability  of  false  10  minute 
prediction  </=  OJO  for  gust 
fronts  with  wind  change  >/=15 
knots 

3)  Storm  motion 

a)  Storm  speed 
accuracy 

±5  knots  for  90%  of  storm 
events  moving  at  >!=  10  knots 

±5  knots  for  90%  of  storm 
events  moving  at  >/=  5  knots 

b)  Storm  direction 

±20  degrees  for  90%)  of  storms 
moving  at  >/=  10  knots 

±20  degrees  knots  for  90%  of 
storms  moving  at  >/=5  knots  and 
±10  degrees  for  50%  of  storms 
moving  at  >/=  5  knots 

4)  Storm  Extrapolated  Position 

a)  Extrapolated 
position  times 

Position  projected  10  minutes 
and  20  minutes  in  advance 

Objective  =  Threshold 
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b)  Extrapolated 
position 
accuracy 


70-minute  extrapolation  within  2  20-minute  extrapolation  within  2 
nm  for  80%  of  storms  moving  at  nm  for  70%>  of  storms  moving  at 
speeds  >  10  knots,  excluding  speeds  >  10  knots  (TRACON 

storms  with  growth,  decay  >/=  2  product) 
levels  (TRACON  product) 


. . . . . . .  . .  . .  1  . .  — — - - 

5)  Storm  cell  information 

a)  Storm  cell 
association 

>/=  90%)  of  features  associated 
to  correct  cell 

>/=  95%  of  features  associated 

to  correct  cell 

b)  Storm  cell 
information 

Threshold  =  Objective 

Identify  80%  of  cells  which  will 
grow  or  decay  by  over  20%  in 
area  in  next  20  minutes 

6)  ASR-9  AP  edit 

a)  Inadvertent  edit 

</=  a  maximum  of  lOkm^  or 

10%  of  contiguous  area  with 
weather  reflectivity  >!-  level  3 

</=  a  maximum  of  lOkm^  or 

10%  of  contiguous  area  with 
weather  reflectivity  >/=  level  2 

b)  Latency 

</=30  seconds  of  ASR'9  update 

Objective  =  Threshold 

c)  Edit  performance 

Edit  70%o  of  AP  when  ASR-9 
level  is  >/=level  3  and  >/=2 
levels  over  actual  reflectivity 
level,  &  AP  >/=  25km^ 

Edit  85%  of  AP  when  ASR-9 
level  is  >/=level  3  and  >/=2 
levels  over  actual  reflectivity 
level,  &  AP  >/=  25km' 

C)  Winds  Products  generation  performance 

1)  Terminal  winds 

a)  Horizontal 
resolution 

±  5  nm  out  to  30  nm  beyond 
TRACON  </=  23000  feet 

±  /  nm  within  TRACON 
boundaries  </=  18000  feet;  5  nm 
elsewhere 

b)  Vertical 
resolution 
(between  levels) 

50  millibars 

25  millibars  <  5000  feet  AGL 
and  </=  1 5  nm  of  the  TDWR 
radar,  50  mb  elsewhere 

c)  Accuracy 

+  10  knots  80%  of  time  in 
regions  and  at  times  when  both 
TDWR  and  NEXRAD  have 
valid  velocity  data 

+5  knots  90%  of  time  m  regions 
and  at  times  when  both  TDWR 
and  NEXRAD  have  valid 
velocity  data 
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3.4.1  Exit  Criteria. 


Products  presented  in  table  3.2-1  must  meet  the 

technical /operational  requirements  as  determined  by  the  Associate 
Program  Manager  for  Test  (APMT) /Integrated  Product  Team  (IPT) 
reflected  in  the  CPP  list  presented  in  table  3.4-1.  The  ITWS 
program  has  identified  the  following  exit  criteria  for  the  DT&E 
and  OT&E  phases: 

DT&E 

a.  ITWS  must  successfully  meet  and  pass  the  technical 
parameters  identified  in  the  system  specification  FAA— E— 
ITWS  ; 

b.  ITWS  must  successfully  demonstrate  interface 
communication; 

c.  ITWS  must  successfully  demonstrate  performance  of  the 
contractor  developed  software  on  data  sets  containing 
various  meteorological  phenomena; 

d.  Successful  Physical  Configuration  Audit  (PCA) /Functional 
Configuration  Audit  (FCA)  must  be  completed; 

e.  Rehosting  of  the  ITWS  onto  new  hardware  must  result  in 
algorithm  performance  that  is  consistent  with  the 
functional  prototype; 

f.  Complete  documentation  of  all  outstanding  change  orders, 
notices,  and  modifications  or  an  approved  plan  for  their 
completion; 

OT&E 

a.  OT&E  regression  testing  must  be  successfully  completed  on 
any  incremental  configuration  updates  during  OT&E  and  on 
the  final  ITWS  hardware/software  configuration.  This 
final  configuration  will  include  all  changes  required  to 
correct  deficiencies  during  OT&E; 

b.  OT&E  Reliability,  Maintainability,  and  Availability  (RMA) 
tracking  must  indicate  that  the  RMA  of  the  ITWS  is 
operationally  satisfactory; 

c.  OT&E  Air  Traffic  (AT)  evaluation  must  be  successfully 
performed  and  all  operational  problems  identified  by  the 
controller  team  must  be  resolved  or  have  an  acceptable 
plan  of  action  in  place; 
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d.  OT&E  Shakedown  Airway  Facilities  (AF)  evaluations  on  the 
ITWS  maintainability  must  be  successfully  performed  and 
all  identified  operational  problems  that  were  identified 
must  be  resolved  or  have  an  acceptable  plan  of  action  in 
place; 

e.  Successful  completion  of  site  adaptability  through 
testing  of  simple  and  complex  sites; 

f.  Satisfy  Pass/Fail  criteria  for  OT&E  requirements  within 
the  VRTM; 

g.  Demonstrate  successful  connection  to  backup  modes 
TDWR/LLWAS . 

3.5  CRITICAL  OPERATIONAL  ISSUES  (COI). 

COIs  address  uncertainty  or  risk  associated  with  an  operational 
system.  They  can  be  categorized  into  two  areas:  (1)  operational 
effectiveness  issues,  reflecting  requirements  of  the  FAA 
operational  user;  and  (2)  operational  suitability  issues, 
reflecting  support  and  maintenance  requirements.  The  ITWS 
program  has  developed  a  structured  approach  to  identify,  manage, 
and  resolve  issues  associated  with  each  COI. 

The  mitigation  of  COIs  is  a  continuing  process  from  prototype 
operations  (DEMVALs)  throughout  the  procurement  lifecycle.  The 
COIs  listed  below  include  those  from  the  ORD  and  the  1994  DEMVAL. 

Section  5.2  represents  a  subset  of  these  COIs  specifically 
related  to  the  1994  DEMVAL.  The  DEMVALS  provided  a  data  set  upon 
which  mitigation  of  the  COIs  could  be  initiated.  Results  of  the 
DEMVALs  provide  an  indication  of  COI  status.  While  COIs  are 
stated  as  resolved  for  the  demonstration  phase,  continued  testing 
is  required  to  assure  resolution  into  the  development  phase  when 
the  contractor  development  system  software  is  procured. 

a.  Will  input  sensor  quality  be  adequate? 

This  COI  was  initially  addressed  during  the  1994  DEMVAL  in  MEM 
and  MCO.  Continued  assessment  of  this  COI  is  needed  since  the 
prototype  system  was  not  completely  representative  of  the 
production  system.  Not  all  NAS  interfaces  were  available 
directly  to  the  ITWS.  These  inputs  were  generated  using 
alternate  software  algorithms  which  provided  the  data  not 
available  through  the  NAS  inputs.  Additionally,  analyses  using 
measured  statistics  for  input  availability  will  also  be  utilized. 
Functional  prototypes  at  MEM,  MCO,  and  DFW  will  enable  additional 
data  to  be  collected  to  assess  the  capability  of  key  sensors 
prior  to  delivery  of  the  first  article  ITWS. 
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During  OT&E,  the  availability  and  performance  of  the  input 
sensors  at  the  OT&E  test  sites  will  be  assessed.  This  testing  on 
the  first  article  systems  will  enable  this  COI  to  be  assessed  in 
an  operational  environment. 

Input  sensors  utilized  are  expected  to  be  commissioned  by  the 
time  ITWS  OT&E  occurs,  therefore  the  data  quality,  in  general, 
should  not  be  an  issue.  However,  input  data  quality  will  be 
monitored  to  ensure  data  integrity.  The  sensor  commissioning 
schedules  will  be  monitored  to  ensure  only  sites  with 
commissioned  sensors  are  utilized. 

Inherent  limitations  of  input  sensor  data  (Anomalous  Propagation 
[AP],  cone  of  silence,  etc.)  will  be  compensated  for  by  utilizing 
available  sensors  which  provide  the  most  accurate  data  for  a 
given  condition.  For  example,  TDWR  reflectivity  will  be  used  for 
the  5~mile  range  (where  ASR“9  tends  to  be  less  sensitive)  and 
NEXRAD  will  be  used  to  edit  ASR-9  AP. 

Utilizing  this  approach  helps  mitigate  the  sensor  quality  issue. 
The  inherent  limitations  will  be  tested  both  in  the  DT&E  and  OT&E 
phases.  Test  scenarios  will  be  developed  to  examine  both  raw 
base  data  for  a  given  sensor  (e.g.,  raw  ASR-9  data)  and  the 
compensated  data  to  verify  the  improved  quality.  Also,  AT 
personnel  will  provide  a  qualitative  evaluation  for  this  COI. 

b.  Will  the  algorithms  and  hardware  function  properly  in  a  large 
TRACON  environment,  with  multiple  TDWRs ,  ASR-9s,  and  NEXRADs? 

Assessment  of  this  COI  will  be  conducted  through  evaluating  the 
ITWS  at  complex  sites  utilizing  multiple  sensor  inputs.  The 
proposed  test  strategy  defined  throughout  section  5.  of  this 
document  will  ensure  that  this  COI  is  addressed. 

The  functional  prototype  at  DFW  in  1995  will  provide  background 
to  evaluate  this  COI,  Multiple  ASR-9  data  archived/recorded  in 
1993,  by  MIT/LL  (for  DFW)  will  also  be  analyzed.  The 
functionality  of  the  hardware  and  algorithms  will  then  be  tested 
by  the  contractor  during  DT&E,  using  the  GFE  algorithms  and 
contractor's  implementation  of  the  software.  Because  the 
algorithms  are  provided  as  GFE  to  the  contractor,  these 
algorithms  will  be  used  as  the  design  reference.  The  contractor 
will  be  required  to  achieve  the  same  probability  of  detections 
(POD),  probability  of  false  alarms  (PFA),  etc.,  that  have  been 
defined  in  the  ORD.  The  DT&E  and  OT&E  testing  performed  will 
test  these  requirements .  The  phased  approach  of  OT&E 
specifically  tests  the  multisensor  environment  using  a 
combination  of  sensors  and  test  drivers  initially  at  the  FAA 
Technical  Center  and  then  at  the  simple  and  complex  site 
configurations.  This  testing  will  provide  data  to  assess  the 
COI , 


20 


c.  Can  the  effectiveness  demonstrated  at  the  DEMVAL  locations  be 
achieved  at  other  ITWS  airports  given  regional  climatic 
differences,  diverse  airport  equipage,  and  availability  of 
inputs? 

In  order  to  begin  addressing  this  COI  data  is  continuing  to  be 
collected  on  the  accuracy  and  operational  effectiveness  at  the 
ITWS  DEMVAL/operational  sites  (operated  and  maintained  by 
MIT/LL) . 

Additionally,  data  obtained  at  DFW  during  the  summer  of  1995  will 
be  collected  and  analyzed  by  ACT-320  to  continue  to  assess  the 
effectiveness  of  the  ITWS.  This  data  set  will  be  an  additional 
point  to  evaluate  the  robustness  of  the  ITWS  algorithms  and  their 
effectiveness  in  different  regional  climates. 

Thereafter,  data  sets  from  other  representative  climate  areas  and 
diversely  eguipped  airports  will  be  collected  and  assessed  for 
robustness  as  described  in  the  paragraph  below.  Data  will  be 
recorded  at  major  TDWR-eguipped  airports  in  the  northeast  and 
upper  midwest  during  convective  seasons  to  complement  the 
Florida,  mid-south,  and  southwest  climatic  data  from  the 
functional  prototype  sites. 

During  the  contractor  conducted  DT&E,  data  sets  representing  the 
diverse  climates  and  equipment  will  be  tested  on  the  contractor 
implemented  software.  Additionally,  the  validity  of  the 
algorithms  will  be  verified.  This  verification  will  be  conducted 
through  two  methods.  First,  the  science  panel  (consisting  of 
meteorological  experts  from  various  scientific  communities)  will 
reconvene  to  assess  the  meteorological  accuracy  of  the  science 
behind  each  algorithm.  Secondly,  a  meteorological  group 
consisting  of  representatives  from  organizations  such  as  ACT-320, 
AOS-250,  and  National  Severe  Storms  Laboratory  (NSSL)  will 
evaluate  algorithm  performance  against  expected  outcome  by  the 
ITWS.  This  will  be  accomplished  through  the  use  of  various 
product  sets  and  conducted  throughout  the  development  test  phase. 

At  the  FAA  Technical  Center,  live  interfaces  will  be  introduced 
to  further  test  the  effectiveness  of  the  algorithms,  which  will 
be  followed  by  simple  and  complex  site  as  part  of  Operational  and 
Shakedown  testing.  OT&E  using  initial  ITWS  articles  will  be 
carried  out  at  two  sites  which  are  different  from  those  selected 
for  functional  prototype  testing  and  recorded  data  evaluations . 

d.  Can  the  ITWS  aid  in  maintaining  effective  airport  capacity 
during  adverse  weather  conditions? 


This  COI  will  be  addressed  by  demonstrating  the  improved 
effective  capacity  (and  reduced  delays)  provided  by  the  initial 
ITWS  operating  capability  at  operational  airports.  Demonstration 
OT&E  testing  using  functional  prototypes  at  MEM  and  MCO 


demonstrated  that  operational  procedures  to  utilize  the  ITWS 
products  have  the  capability  to  improve  effective  capacity. 
Additional  evaluations  using  functional  prototypes  at  MEM,  MCO, 
and  DFW  will  enable  additional  data  to  be  collected  and  the 
effect  on  capacity  provided  by  the  initial  ITWS  to  be  evaluated. 
This  will  occur  prior  to  delivery  of  the  first  article. 

ACT-320  will  perform  analysis  on  data  provided  by  MIT/LL  and  AT 
on  airport  capacity  prior  to  ITWS  installation  at  the  OT&E 
airports  and  post— ITWS  installation.  Before  OT&E  commences, 
airport  capacity  data  for  the  sites  selected  will  be  collected 
and  evaluated.  OT&E  will  also  use  some  qualitative  analyses 
techniques  (i.e.,  questionnaires)  to  augment  the  quantitative 
data  analyses . 

e.  Are  the  ITWS  products  usable  without  the  need  for 
meteorological  interpretation? 

This  COI  was  evaluated  during  the  Demonstration  OT&E  testing 
using  functional  prototypes  in  MEM  and  MCO.  Results  of  this 
evaluation  indicated  that  controllers  did  not  require 
meteorological  interpretation  of  the  products.  Evaluations  of 
ITWS  prototype  operations  in  MCO,  MEM,  and  DFW  in  addition  to 
evaluations  conducted  during  OT&E  will  enable  continued 
assessment  of  this  COI.  A  strong  training  program  will  be 
conducted  to  ensure  that  end  users  fully  understand  the  system 
and  system  functionality. 

f.  Is  the  ITWS  resilient  under  loss  of  input  from  interfaced 
svstems/sensors  e.o..  TDWR.  ASR-9 .  Remote  Maintenance  Monitoring 
System  (RMMSl.  and  NEXRAD? 

The  resiliency  of  the  ITWS  will  be  evaluated  during  OT&E. 
Resiliency  is  being  defined  as  the  system's  ability  to  recover  to 
its  full  operational  functionality  when  inputs  from  interfaced 
systems/sensors  are  lost  and  subsequently  restored,  within  an 
operationally  acceptable  timeframe.  Table  3.2-1  in  the  ITWS  ORD 
addresses  the  impact  on  specific  ITWS  products  upon  input  sensor 
failure . 

This  COI  was  initially  addressed  during  the  Demonstration  Phase 
OT&E  testing  at  MEM  and  MCO  in  1994.  As  with  other  COIs,  the_ 
DT&E  testing  using  data  sets  and  specification  requirements  will 
further  mitigate  the  risk  posed  by  this  COI.  Specific  OT&E 
testing  will  be  conducted  at  the  FAA  Technical  Center  and  at  the 
operational  sites  to  further  evaluate  this  COI.  AF  personnel 
will  conduct  testing  on  RMA  of  the  ITWS  system  and  its  supporting 
sensors  during  shakedown  testing. 
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g.  Are  the  ITWS  products  suitable  for  AT  use? 

Previous  Demonstration  Phase  OT&E  testing  of  functional 
prototypes  in  MEM  and  MCO,  indicated  that  the  ITWS  products  are 
suitable  for  AT  use.  Additionally,  the  ITWS  products  have  been 
developed  in  conjunction  with  the  ITWS  User  Group. 

To  further  evaluate  this  COI,  an  assessment  of  the  ITWS  products 
using  contractor  implemented  software  will  be  performed.  The 
testing  methodology  (questionnaires,  observations,  and 
interviews)  will  be  similar  to  that  during  previous  DEMVALS.  The 
testing  will  be  conducted  by  ACT-320  and  the  data  collected  from 
questionnaires,  observations,  and  interviews  will  provide 
information  necessary  to  address  this  COI .  AT  representatives 
from  various  regions  and  facilities  (TRACONs  and  Air  Route 
Traffic  Control  Centers  (ARTCC))  will  travel  to  the  DT&E  and  the 
phased  OT&E  simple  and  complex  sites  to  utilize  and  evaluate  the 
ITWS  and  support  the  overall  testing  effort. 

h.  Does  the  ITWS  meet  the  critical  performance  threshold 
requirements  of  the  ORD? 

The  status  of  this  COI  from  the  1994  DEMVAL  report  is  partially 
resolved.  The  prototype  will  continue  to  be  monitored  during 
1995  at  DFW,  MEM, and  MCO  with  additional  analyses  performed.  The 
MIT/LL  "Integrated  Terminal  Weather  System  (ITWS)  Demonstration 
and  Validation  Operational  Test  and  Evaluation"  report  statistics 
will  also  be  examined,  and  DT&E  data  sets  will  further  validate 
the  threshold  requirements.  During  OT&E  using  meteorological  and 
statistical  analysis  output  data  will  be  compared  with  raw  sensor 
data  to  verify  product  accuracy. 

The  scientific  community  accepts  this  method  as  a  valid  process 
to  assess  truth.  The  meteorological  validation  team  will  provide 
further  independent  assessment  to  assure  resolution  to  this  COI . 

Additionally,  the  following  testing  and  oversight  will  occur  to 
certify  the  implementation  of  the  contractor  developed  software 
algorithms : 

The  Science  Panel  will  be  reconvened  to  assure  the  science 
behind  the  ITWS  Algorithms  remains  valid; 

An  independent  group  will  oversee  the  meteorological 
verification  of  the  ITWS  algorithms  using  various  product  sets 
and  assess  the  process  throughout  the  Development  Test  phase, 
and; 

The  "A"  Specification  will  require  discrete  points  in  the 
software  that  will  allow  the  contractor  and  the  government  to 
inject  data  to  prove  the  contractor's  implementation  of  the 
GFE  algorithms  give  the  required  results. 
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AF  pGrsonnsl  from  AOS~250  will  also  be  involved  during  the 
DT&E  and  OT&E  testing  to  assure  this  COI  continues  to  be 
resolvedo  The  above  efforts  should  continue  to  assure  this 
COI  is  adequately  addressed. 

1.  Do  the  ITWS  products  enhance  the  effectiveness  of  traffic 
plannina/manaoement  (delays,  airport  acceptance  rate^ — 
flow,  etc.1  during  adverse  conditions  in  the  terminal  area7 — Are 
terminal  airspace  and  runways  used  more  effectively? 

This  COI  was  initially  addressed  during  the  1994  DEMVAL  at  MEM 
and  MCO  where  OT&E  testing  using  functional  prototypes 
demonstrated  that  operational  procedures  to  utilize  the  ITWS 
products  improved  effective  capacity.  Additional  input  using 
functional  prototypes  at  MEM,  MCO,  and  DEW  will  provide  additional 
data  to  further  examine  the  effective  capacity  provided  by  the 
initial  ITWS  prior  to  delivery  of  the  first  article.  Data  to 
address  this  COI  for  OT&E  will  be  obtained  from  pre-ITWS  air 
traffic  measures  and  questionnaire  evaluations.  See  item  d. 

i .  Is  the  ITWS  display  visible  under  anticipated  lighting 
conditions? 

This  COI  will  be  resolved  by  means  of  specification  requirements 
and  system  use  within  the  operational  environment. 

k.  Does  the  ITWS  reduce  (perceived)  controller  workload  during 
adverse  weather  conditions  in  the  terminal  area? 

The  results  of  the  workload  scale  administered  to  air  traffic 
personnel  in  conjunction  with  previous  Demonstration  Phase  0T6E 
testing  in  MEM  and  MCO,  indicated  that  perceived  workload  was 
reduced  during  adverse  weather  conditions.  The  results  of  the 
workload  scale  are  contained  in  the  "ITWS  Demonstration/ 
Validation  Phase  OT&E  Final  Report,  (Draft)." 

Additional  workload  data  will  be  collected  in  conjunction  with 
the  prototype  evaluation  at  DFW  in  1995,  to  further  verify 
results  obtained  in  1994.  A  workload  scale  will  be  administered 
at  the  OT&E  Integration  and  Operational  testing  at  the  simple  and 
complex  sites. 

3.6  MINIMUM  ACCEPTABLE  OPERATIONAL  PERFORMANCE  REQUIREMENTS 
( MAOPR ) . 

Air  Traffic  Plans  and  Requirements  Service  (ATR-134.1)  has 
identified  the  ITWS  MAOPRs  requirements;  they  correspond  to  the 
threshold  values  of  the  CPPs  in  table  3.4-1. 
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4.  TEST  &  EVALUATION  (T&E^  PROGRAM  MANAGEMENT. 

4 . 1  MANAGEMENT . 

The  following  subsections  identify  the  roles  and  responsibilities 
for  the  organizations  involved  in  the  ITWS  T&E  process. 

4.1.1  NAS  Configuration  Control  Board  (CCB). 

a.  Approves  DT&E  and  PAT&E  requirements  contained  in  the 
project  specification  (e.g.,  project  specification  VRTM) . 

b.  Approves  test  standards  and  definitions. 

c.  Approves  NAS-SS-1000  NCPs  and  IRDs  that  affect  system 
requirements • 

4.1.2  Test  Policy  Review  Committee  (TPRC) . 

a.  Supports  T&E  policy,  test  standards,  and  definitions. 

b.  Approves  TPRC  operating  procedures. 

c.  Approves  FAA  TEMP  and  revisions. 

d.  Approves  test  policy  waivers. 

e.  Resolves  disagreements  on  T&E  issues  when  agreements 

cannot  be  reached  at  lower  levels  of  FAA  management. 


4.1.3  Aviation  Weather  Development  Program,  AND-4  60, _ ITWS 

Program  Manager  (PM) . 

a.  Responsible  for  overall  program  management. 

b.  Presents  T&E  deployment  issues  to  the  Deployment 
Readiness  Review  (DRR) . 

c.  Arranges  with  APMT  for  T&E  support,  coordination  and 
monitoring  through  an  annual  Program  Directive  (PD). 

d .  Approves  PD . 

e.  Tasks  APMT  to  prepare  PDs  between  the  program  office 
and  other  FAA  organizations. 

f.  Requests  funding  for  project  T&E  which  is  included  in 
the  overall  program  funding. 

g.  Responsible  for  receiving  TPRC  approval  for  the  FAA 
TEMP. 

h.  Prepares  test  policy  waiver  requests,  and  submits  them 
to  the  TPRC  Secretariat. 

i.  Coordinates  T&E  requirements  for  Department  of  Defense 
(DOD) ,  or  other  government  agencies,  on  joint 
procurement,  as  the  project  requires. 

j.  Develops,  or  has  the  APMT  develop,  the  project 
specification  VRTM,  and  incorporates  these  requirements 
into  the  project. 

k.  With  APMT  support,  brings  unresolved  T&E  issues  before 
the  TPRC  via  the  TPRC  Secretariat. 

l.  Approves  DT&E  test  plans,  procedures,  and  reports. 

m.  Reviews  DT&E  test  plans,  procedures,  and  reports. 


n.  Recommends  to  the  contract  officer  (CO)  approval  of 
DT&E  test  plans,  procedures,  and  reports. 

o.  Monitors  DT&E  contractor  conducted  testing. 

p.  Reviews  OT&E  Integration  and  OT&E  Operational  test 
requirements,  plans,  procedures,  and  reports. 

q.  Approves  OT&E  Integration  and  OT&E  Operational  test 
requirements,  plan,  procedures,  and  reports. 

r.  Monitors  OT&E  Integration  and  OT&E  Operational  tests. 

s.  Monitors  OT&E  Shakedown. 

t.  Reviews  Field  Shakedown  requirements  with  the  Airway 
Facilities  Division  organization. 

u.  Reviews  Site  Acceptance  Test  (SAT)  test  plans, 
procedures,  and  reports. 

V.  Monitors  Field  Shakedown. 

w.  Oversees  distribution  for  DT&E/SAT  test  plans, 

procedures,  and  reports. 

X.  Responsible  for  FAA  TEMP  distribution. 

y.  Responsible  for  identifying  and  prescribing  appropriate 
distribution  and  accountability  controls  for  program 
technology  that  is  critical. 

2.  Prepares  WCPs  for  designated  test  locations. 

[  NAS  Development  Special  Assistant  (AND-3). 

a.  Member  of  TPRC. 

b.  Reviews  FAA  TEMP. 

c.  Supports  development  of  revisions  to  test  policy,  test 
standards,  and  definitions. 


1 . 5  MAS  Transition  and  Implementation  Service  (AMS 


a.  Member  of  TPRC. 

b.  Provides  supportable  requirements  to  the  APMT  for 

inclusion  in  the  FAA  TEMP,  which  serves  as  guidance  to 
AOS  for  the  OT&E  plans. 

c .  Reviews  FAA  TEMPs . 

d.  Reviews  requirements,  plans,  and  procedures  for  OT&E 
plans , 

e.  Provides  personnel  for  conducting  and/or  monitoring  the 
conduct  of  OT&E  Shakedown. 

f .  Reviews  OT&E  Shakedown  reports . 

g.  Reviews  PDs . 

h .  Approves  PDs . 


4.1.6  Communication.  Navigation  and  Surveillance  Enqineerin 


Test  Division  (ACT-300). 


a.  Member  of  the  TPRC. 

b.  Provides  APMT. 

c.  Prepares  project  TEMP. 

d.  Reviews  test  plans. 
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e.  Reviews  DT&E,  OT&E  Integration  and  OT&E  Operational 
test  requirements. 

f.  Provides  concurrence  on  OT&E  Integration  and  OT&E 
Operational  test  plans  and  reports  prior  to  review. 

g.  Presents  unresolved  T&E  issues,  significant  T&E  test 
result  problems,  or  violations  of  T&E  policy  to  the 
TPRC. 

h.  Provides  T&E  assessments  to  the  DRR. 

i.  Provides  for  FAA  Technical  Center  facility  readiness. 

4.1.7  Associate  Program  Manager  for  Test  f ACT-320). 

a.  Supports  development  of  test  policy  and  test  standards. 

b.  Acts  as  the  agent  of  the  PM  to  manage  the  T&E  program; 
including  establishing  overall  test  schedules, 
coordinating  tests,  ensuring  that  all  test  requirements 
are  satisfied,  and  that  tests  are  performed  in 
accordance  with  approved  procedures . 

c.  Prepares,  coordinates,  and  approves,  with  the  PM,  an 
annual  PD  which  addresses  all  T&E  task  support 
activities  and  resources  required  for  the  project. 

d.  Prepares  appropriate  T&E  inputs  to  project 
documentation,  (e.g.,  project  procurement  package)  as 
specifically  tasked  in  the  PD. 

e.  Prepares  PDs  between  the  project  office  and  other  FAA 
or  DOD  organizations  to  fund  and/or  arrange  for  the 
organizations'  participation  in  T&E  activities. 

f.  Jointly  prepares  and  updates  the  FAA  TEMP  with  the  PM. 
Provides  updates  of  available  test  results  during  DRR. 
Reviews  DT&E  test  requirements,  plans,  procedures,  and 
reports . 

i .  Arranges  DT&E  and  PAT&E  test  support . 

j.  Reviews  DT&E  and  PAT&E  test  requirements. 

k.  Coordinates  with  performing  organizations,  and  monitors 
DT&E,  OT&E,  and  PAT&E  activities. 

l.  Reviews  contractor-prepared  DT&E  and  PAT&E  plans, 
procedures,  and  reports. 

m.  Prepares  DT&E  and  PAT&E  test  plans,  procedures,  and 
reports  when  tasked  by  the  PM  to  develop  hardware  or 
software,  instead  of  a  contractor. 

n.  Directs  and  conducts  DT&E  testing  if  tasked  by  the 
PM/Associate  Program  Manager  for  Engineering  (APME)  and 
monitors  DT&E  testing  performed  by  a  contractor. 

o.  Reviews  DT&E  and  PAT&E  requirements  for  inclusion  in 
the  FAA  TEMP. 

p.  Prepares  OT&E  Integration  and  OT&E  Operational  test 
requirements  for  inclusion  in  the  FAA  TEMP. 

q.  Prepares  OT&E  Integration  and  OT&E  Operational  test 
plans,  procedures,  and  reports. 

r.  Reviews  OT&E  Shakedown  requirements,  plans,  and 
procedures . 
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Directs  and  conducts  OT&E  Integration  and  OT&E 
Operational  tests.  AOS“250  may  optionally  participate 
in  test  conduct. 

Reviews  all  OT&E  Shakedown  reports  (information  only). 
Reviews  Field  Shakedown  requirements,  plans, 
procedures ,  and  reports . 

Monitors  OT&E  Shakedown. 

Monitors  Field  Shakedown. 

Air  Traffic  Plans  and  Requirements  (ATR) . 

Member  of  TPRC. 

Provides  requirements  for  and  reviews  the  FAA  TEMP. 
Provides  operational  expertise  and  planning  for 
conducting  and  analyzing  tests. 

Reviews  DT&E,  OT&E,  and  PAT&E  requirements. 

Provides  personnel  to  support  monitoring  and  conduct  of 
DT&E. 

Reviews  program  PDs . 

Provides  test  requirements  via  the  FAA  TEMP,  supports 
test  plan  development,  and  reviews  test  plans  and 
procedures  for  OT&E  Integration  and  OT&E  Operational 
tests. 

Provides  and  approves  additional  test  requirements 
(that  do  not  exceed  OT&E  Shakedown  durations  or  costs 
as  baselined  in  the  FAA  TEMP)  not  identified  in  the 
TPRC-baselined  FAA  TEMP  for  OT&E  Integration  and  OT&E 
Operational  tests.  When  change  or  additions  are 
required  which  exceed  cost  or  schedule  allotments 
previously  planned,  the  normal  process  for  adjusting 
the  planned  testing  and  resolving  disagreements 
applies. 

Determines  the  operational  acceptability  of  new  ATC 
operational  computer  programs  or  systems  prior  to  their 
delivery  for  operational  testing  and  use  in  field 
facilities. 

Provides  personnel  for  conducting  and/or  monitoring  the 
conduct  of  OT&E  Integration  and  OT&E  Operational  tests . 
Reviews  OT&E  Integration  and  OT&E  Operational  test 
reports . 

Provides  and  coordinates  test  requirements,  supports 
test  plan  development,  and  reviews  test  plans  and 
procedures  for  OT&E  Shakedown. 

Provides  personnel  for  conducting  and/or  monitoring  the 
conduct  of  OT&E  Shakedown. 

Reviews  OT&E  Shakedown  reports. 

Provides  and  reviews  requirements,  plans,  and 
procedures  for  Field  Shakedown. 

Monitors  the  conduct  of  Field  Shakedown. 

Reviews  Field  Shakedown  reports . 

Provides  a  deployment  recommendation  based  on  OT&E 
Shakedown  results,  in  support  of  the  DRR. 

Develops  the  ORD. 
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4.1.9  Operational  Support  Service  (AOS). 

a.  Member  of  TPRC. 

b.  Identifies  and  develops  with  the  PM  and  APMT,  OT&E 
Shakedown  requirements  for  inclusion  in  the  FAA  TEMP. 

c.  Optionally  supplies  draft  PD,  reviews,  and  approves 
final  PD. 

d.  Reviews  FAA  TEMP. 

e.  Reviews  OT&E'  Integration  and  OT&E  Operational  test 
requirements,  plans,  and  reports. 

f.  Monitors  DT&E  tests. 

g.  Monitors  OT&E  Integration  and  OT&E  Operational  tests, 
and  optionally  participates  in  OT&E  Integration  and 
OT&E  Operational  test  conduct. 

h.  Prepares  OT&E  Shakedown  requirements,  plans, 
procedures,  and  reports  in  coordination  with  ATR. 

i.  Approves,  in  coordination  with  ATR,  additional  OT&E 
Shakedown  requirements  that  do  not  exceed  OT&E 
Shakedown  durations  or  costs  as  baselined  in  the  TEMP. 

j.  Approves  OT&E  Shakedown  plans,  procedures,  and  reports. 

k.  Directs  and  conducts  OT&E  Shakedown  as  applicable  to 
OT&E  requirements.  ATR  will  support  and  participate  in 
those  tests  that  are  applicable  to  ATR  OT&E  Shakedown 
requirements . 

l .  Provides  personnel  for  performing  and/or  monitoring  the 
conduct  of  OT&E  Shakedown. 

m.  Conducts  OT&E  Shakedown  data  analysis. 

n.  Provides  a  deployment  recommendation  based  on  OT&E 
Shakedown  results  in  support  of  the  DRR. 

o.  Monitors,  and  optionally  participates,  in  test  conduct 
of  Field  Shakedown. 

4.1.10  NAS  System  Analysis  and  Integration  Division  (ASD-120). 

a.  Reviews  FAA  TEMP. 

b.  Provides  the  NAS-SS-1000  System  Specification 
requirements  for  inclusion  in  the  FAA  TEMP  VRTM,  or 
coordinates  requirements  for  those  projects  not 
included  in  the  NAS-SS-1000. 

c.  Provide  inputs  to  mission  needs  analysis  that  serve  as 
the  basis  for  various  Key  Decision  Points. 

d.  Provide  inputs  or  revised  engineering  documentation 
(specifications.  Statements  of  Work,  NCPs,  TEMPs,  etc.) 
for  conformance  with  system  engineering  policies, 
standards,  and  baseline  specifications. 

4.1.11  System  Engineering  Management  (ASP- 140). 

Serves  as  TPRC  Secretariat. 

Formulates  revisions  to  test  policy,  test  standards  and 
definitions  for  consideration  and  endorsement  by  the 
TPRC. 
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c.  Verifies  compliance  with  FAA  Order  1810. 4B  and 
standards . 

d.  Develops  and  maintains  the  TPRC  Operating  Procedures. 

e.  Provides  and  maintains  implementation  traceability  for 
WAS  Verification  via  the  VRTMs  contained  in  the  NAS-SS- 
1000  System  Specification. 

f.  Develops  VRTMs  for  new  WAS-SS-1000  System  Specification 
projects  and  NAS  IRDs. 

4.1.12  Office  of  Independent  Operational  Test  &  Evaluation 

(ATQ)  . 

a.  Member  of  TPRC. 

b.  Provides  independent  oversight  of  all  ITWS  testing 
efforts . 

c.  Assesses  operational  suitability  and  effectiveness  of 
the  ITWS  system. 

d.  Co-approves  the  TEMP. 

e.  Reviews  and  comments  on  DT&E  and  OT&E  plans, 
procedures,  and  reports. 

f.  Provides  operational  readiness  assessment  reports  to 
the  FAA  Administrator. 

g.  Responsible  for  Independent  OT&E. 

4.1.13  Director  of  Acquisitions  fASU) . 

a.  Member  of  TPRC. 

b.  Reviews  and  approves  PDs . 

c.  Reviews  FAA  TEMP  and  contractor's  MTP. 

d.  Reviews  DT&E  test  plans,  procedures,  and  reports. 

e.  Reviews  PAT&E  test  plans,  procedures,  and  reports. 

f.  Verifies  completeness  of  program  by  reviewing  the  final 
OT&E  I/O  testing.  Shakedown  and  Field  Shakedown  reports 
from  each  site. 

g.  Provides  Associate  Program  Manager  for  Quality  (APMQ) 
and  Quality  Reliability  Officer  (QRO) . 

4.1.14  Office  of  Air  Traffic  System  Management  (ATM). 

a.  Reviews  Field  Shakedown  requirements,  plans, 
procedures ,  and  reports . 

b.  Determines  the  operational  acceptability  of  new  ATC 
operational  computer  programs  or  systems  prior  to  their 
delivery  for  operational  testing  and  use  in  field 
facilities . 

c.  Monitors  OT&E  Operational  testing. 

d.  Monitors  Field  Shakedown. 

e.  Monitors  computer  program  implementation  schedules  to 

ensure  operational  requirements  are  met. 

f.  Manages  requirements  for  new  airspace  management 
systems . 

g.  Reviews  PDs  via  ATR. 


30 


4.1.15  Air  Traffic  Rules  and  Procedures  Service  (ATP). 

a.  Reviews  Field  Shakedown  requirements,  plans, 
procedures,  and  reports. 

b.  Monitors  Field  Shakedown. 

c.  Develops  procedures  for  system  implementation. 

d.  Reviews  PDs  via  ATR. 

4.1.16  Communications/Infrastructure  (ACT-330). 

a.  Conduct  RMS/MPS  interface  testing  during  OT&E 
Integration. 

b.  Develops  plans,  reports,  and  provides  overall 
coordination  for  the  RMS  effort. 

4.1.17  FAA  Contracting  Officer. 

a.  Approves  DT&E  and  PAT&E  test  plans,  procedures,  and 
reports  for  contractual  compliance. 

b.  Ensures  DT&E  tests  are  conducted  per  contract. 

4.1.18  Regional  Air  Traffic  Division. 

a.  Support  PM  via  ATR  in  development  of  test  requirements 
for  inclusion  in  the  FAA  TEMP. 

b.  Supports  PM  in  implementation  of  FAA  TEMP  at  test  and 
operational  facilities,  as  required  by  ATR. 

c.  Supports  AF  Division  in  the  development  of  Field 
Shakedown  requirements,  plans,  procedures,  and  reports, 
with  the  inclusion  of  Regional  AT  Division  objectives 
and  interests . 

d.  Provides  coordination  to  AF  Division  for  Field 
Shakedown  requirements,  plans,  procedures,  and  reports. 

e.  Participates  in  the  conduct  of  OT&E  Integration  and 
OT&E  Operational  testing,  and  OT&E  Shakedown,  as 
coordinated  with  the  ATR  organization. 

f.  Supports  Field  Shakedown  that  is  in  satisfaction  of 
Regional  AT  Division  test  requirements  or  objectives, 
as  coordinated  with  AF  Division. 

g.  Conducts  Field  Shakedown  in  coordination  with  AF 
Division . 

h.  Monitors  Field  Shakedown. 

i.  Reviews  PD  via  ATR. 

4.1.19  Air  Traffic  Facilities. 

a.  Participates  in  FAA  TEMP  activities  as  required  by  ATR 
through  Regional  AT  Division. 

b.  Supports  development  of  Field  Shakedown  requirements, 
plans,  procedures,  and  reports,  in  coordination  with 
facility  AF  organizations. 
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c.  Conducts  and  monitors  Field  Shakedown  and  reports 
results  in  coordination  with  facility  AF  organizations 
and  Regional  AT  Division. 

d.  Reviews  PD  via  ATR  and  Regional  AT  Division. 


4.1.20 


Ti onal  Airway  Facilities  Division 


a .  Supports  PM  in  development  of  test  reguirements  for 
inclusion  in  FAA  TEMP. 

b.  Supports  PM  in  implementation  of  FAA  TEMP  at  test  and 
operational  facilities. 

Co  Responsible  for  overall  Field  Shakedown^  in  cooperation 
with  AT  Division. 

^  Qo~approves,  jointly  with  AT  Division,  Field  Shakedown 
requirements  with  the  PM. 

e.  Approves  Field  Shakedown  plans,  procedures,  and 
reports . 

f.  Participates  in  the  conduct  of  OT&E  Integration  and 
OT&E  Operational  testing,  and  OT&E  Shakedown,  as 
coordinated  with  AOS . 

g.  Directs  Field  Shakedown  that  is  in  satisfaction  of  AF 
Division  test  requirements  or  objectives,  and  as 
coordinated  with  AT  Division. 

h.  Conducts  Field  Shakedown  in  coordination  with  AT 
Division.  AOS-250  have  the  option  of  participating  in 
test  conduct. 


Airway  Facilities  Sectors. 


a.  Participates  in  FAA  TEMP  activities  as  required  by  AF 
Division . 

b.  Develops  Field  Shakedown  requirements,  plans,  and 
procedures,  in  coordination  with  facility  AT 
organization . 

c.  Conducts  Field  Shakedown,  including  Joint  Acceptance 
Inspection  (JAI),  and  reports  results  in  coordination 
with  facility  AT  organization. 


, 1.22  Test  Plan  Working  Grc 


a.  Meet  periodically  to  discuss  test  related  issues 
concerning  the  TEMP,  DEMVALs,  OT&E  schedules,  and  other 
related  issues. 

b.  Provide  input  for  test  requirements  and  represent 
respective  organizations  regarding  acceptance  of  test 
responsibilities  and  input  for  test  requirements. 

c.  Provides  input  into  the  preparation  of  test  plans  and 
test  requirements . 
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4.1.23  MIT/LL. 


a.  Provide  support  to  the  ITWS  prime  contractor  as 
requested  by  AND-460. 

b.  Provide  technical  support/meteorological  background  on 
algorithms . 

c.  Provide  support  to  AND-460  and  ACT-320  during  overall 
test  effort  on  the  technical  and  scientific  background 
of  the  ITWS  during  the  development  and  production 
phase . 

d.  Operate  ITWS  functional  prototypes  in  support  of 
Development  OT&E  Testing  at  MEM,  MCO,  and  DFW. 

4.2  INTEGRATED  SCHEDULE. 

Appendix  B  contains  the  ITWS  Integrated  Test  Schedule  which 
details  the  sequential  relationship  of  all  T&E  events  and 
milestones  relative  to  the  key  acquisition  events  of  the  program. 
System  development  will  begin  in  the  1996  timeframe  with  OT&E 
beginning  in  1999.  The  schedule  will  be  updated  as  confirmed 
dates  become  available. 

Key  milestones  of  the  schedule  are  the  Preliminary  Design  Review 
(PDR),  Critical  Design  Review  (CDR) ,  DT&E,  Test  Readiness  Review 
(TRR) ,  Contract  Acceptance  Inspection  (CAI),  and  first 
Operational  Readiness  Demonstration  (ORD) .  The  PDR  is  an  early 
opportunity  for  the  FAA  to  examine  the  high-level  design  of  the 
contractor  developed  system.  The  contractor  will  present  the 
detailed  design  to  the  FAA  at  the  CDR.  At  this  point,  the 
program  can  move  into  the  software  development  phase.  The  DT&E 
phase  will  present  the  FAA  with  a  series  of  opportunities  to 
monitor  development.  DT&E  will  commence  immediately  after 
successful  TRR.  The  TRR  will  indicate  the  preparedness  for  the 
Factory  Acceptance  Testing  (FAT) ,  which  will  transition  the 
system  from  the  factory  to  an  operational  site.  The  system 
transition  of  ownership  from  the  contractor  to  the  FAA  occurs  at 
CAI.  The  first  ORD  is  part  of  Field  Shakedown  Testing  and 
precedes  system  commissioning. 

4.3  ITWS  TEST  AND  EVALUATION  FUNDING. 

The  funding  profile  estimated  to  support  test  and  evaluation 
through  fiscal  year  2001  is  presented  in  table  4.3-1. 
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TABLE  4.3-1.  ITWS  TEST  AND  EVALUATION  FUNDING 
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4 . 4  TEST  PLANS . 


Table  4.4-1  lists  the  Development  Phase  test  plans,  expected 
completion  dates,  and  responsible  organizations. 

TABLE  4.4-1.  DEVELOPMENT  PHASE  TEST  PLANS 


Developinent  Phase  Test 

Plans 

Document  Title 

Expected 

Completion 

Date 

Responsible  Organization 

FAA  TEMP 

Apr-95 

ACT-320 

Human  Factors  Plan 

Mar-95 

ACT-320 

FAA  OT&E  Operational  and 
Integration  Test  Plan 

Jun-99 

ACT-320 

OT&E  Shakedown  Test  Plan 

Jul-99 

AOS-200 

lOT&E  Plan 

TBD 

ATQ/AT 

Contractor's  Master  Test  Plan 
(CMTP) 

TBD 

Development  Contractor 

Software  Test  Plan  (includes 
DQT/FQT) 

TBD 

Development  Contractor 

DT&E  FAT  Plan 

TBD 

Development  Contractor 

DT&E  SAT  Plan  (includes  Delta 
DT&E  Test  requirements) 

TBD 

Development  Contractor 

PAT&E  FAT  Plan 

TBD 

Development  Contractor 

PAT&E  SAT  Plan 

TBD 

Development  Contractor 

Reliability  Test  Plan 

TBD 

Development  Contractor 

Maintainability  Test  Plan 

TBD 

Development  Contractor 
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4.4.1  Government  Test  Plans. 


A  set  of  test  plans  will  be  developed  by  the  organizations 
responsible  for  each  T&E  phase  identified  within  this  TEMP.  Test 
plans  define  the  range  of  tests  to  be  performed,  input  data, 
initialization  requirements,  expected  output,  qualitative  methods 
and  criteria  for  evaluating  test  results . 

4. 4. 1.1  FAA  TEMP. 

The  FAA  TEMP  is  written  by  the  government  and  is  in  accordance 
with  FAA-STD-024b,  1810.4b,  and  1810.  If.  This  document  details 
the  overall  test  philosophy  throughout  the  DT&E,  OT&E,  and  PAT&E 
test  phases. 

4.4. 1.2  Human  Factors  Plan. 

The  Human  Factors  Plan  is  prepared  by  the  government  and  outlines 
the  proposed  approach  and  methodology  utilized  to  address  human 
engineering  issues  throughout  the  acquisition  cycle.  This  plan 
will  drive  all  test  and  evaluation  efforts  regarding  operator 
performance  and  display  issues. 

4. 4. 1.3  FAA  OT&E  Operational  and  Integration  Test  Plan. 

This  plan  is  prepared  by  the  government  and  is  used  to  ensure 
that  the  NAS  requirements  are  thoroughly  tested.  It  will  also 
test  the  system  to  verify  that  interfaces  between  existing  NAS 
systems  are  not  degraded  due  to  the  introduction  of  the  new 
system.  The  plan  will  also  address  how  operational  effectiveness 
and  suitability  of  the  system  will  be  evaluated. 

4. 4. 1.4  OT&E  Shakedown  Test  Plan. 

The  OT&E  Shakedown  Test  Plan  defines  the  OT&E  testing  within  the 
operational  environment.  This  testing  will  also  verify  the 
readiness  of  personnel  and  procedures  with  respect  to  the  system. 

4.4.2  Contractor  Test  Plans. 

The  contractor  will  develop  a  series  of  test  plans  to 
successfully  test  the  system  through  the  DT&E  and  PAT&E  test 
phases . 
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4. 4. 2.1  Contractor  Master  Test  Plan  (CMTP). 

The  CMTP  defines  the  overall  test  philosophy  of  the  contractor 
and  summarizes  tests  required  to  be  conducted  by  the  contractor 
in  the  DT&E  and  PAT&E  test  phases.  It  details  the  methods  for 
implementing  and  controlling  the  various  testing  programs. 

4. 4. 2. 2  Software  Test  Plan  (STP). 

The  STP  defines  the  scope  of  testing  to  be  conducted  at  the 
software  development  level.  It  will  provide  plans  for  unit 
tests,  computer  software  components  and  computer  software 
configuration  items.  The  testing  will  validate  the  major  ITWS 
system  components  of  communications,  processing  and  control. 
Additionally,  this  plan  will  define  software  test  cases  and  post¬ 
conduct  analysis  to  determine  the  effectiveness  of  the  GFE 
(meteorological  algorithms).  The  software  test  plan  includes 
planned  testing  for  DT&E  DQT  and  DT&E  FQT. 

4. 4. 2. 3  DT&E  Factory  Acceptance  Test  (FAT)  Plan. 

The  DT&E  FAT  Plan  is  prepared  by  the  contractor  and  details  the 
pre-delivery  system  testing  to  be  conducted  by  the  contractor 
within  the  their  facility.  This  testing  will  be  conducted  on  the 
first  article  systems.  This  plan  will  contain  the  pass/fail 
criteria  for  the  specification  requirements. 

4. 4. 2. 4  DT&E  SAT  Plan. 

The  DT&E  SAT  Plan  is  prepared  by  the  contractor  and  will  identify 
the  testing  to  be  conducted  at  the  first  article  site  location. 
This  plan  will  include  the  Delta  DT&E  test  requirements  which 
will  incorporate  tests  that  were  unable  to  be  accomplished  at  the 
contractor's  facility. 

4. 4. 2. 5  Production  Acceptance  Test  &  Evaluation  (PAT&E)  FAT 
Plan . 

The  PAT&E  FAT  Plan  is  prepared  by  the  contractor  and  details  the 
test  methodology  on  each  production  unit  to  be  conducted  at  the 
contractor's  facility.  This  will  ensure  that  the  production 
units  meet  the  same  requirements  of  the  first  article  systems 
from  previous  test  phases. 

4. 4. 2. 6  PAT&E  SAT  Plan. 

The  PAT&E  SAT  Plan  is  prepared  by  the  contractor  and  describes 
the  testing  for  each  production  unit  to  be  conducted  at  the 
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system  delivery  sites.  This  will  ensure  that  the  production 
units  meet  the  same  requirements  as  those  sent  to  the  first 
operational  sites. 

4. 4. 2. 7  Reliability  Test  Plan. 

The  Reliability  Test  Plan  is  prepared  by  the  contractor  in 
accordance  with  MIL~STD“785 .  This  plan  assures  that  the 
reliability  values  and  requirements  of  FAA-E-ITWS  and  the  SOW  are 
appropriately  tested. 

4. 4. 2. 8  Maintainability  Test  Plan. 

The  Maintainability  Test  Plan  is  prepared  by  the  contractor  in 
accordance  with  MIL-STD-470B .  This  plan  assures  that  the 
requirements  of  FAA-E-ITWS  and  the  Statement  of  Work  (SOW)  are 
appropriately  tested. 

4.5  TEST  PROGRAM  RESOURCES. 

4.5.1  Manpower  and  Training. 

The  ITWS  test  team  includes  AT,  AF ,  AND-460,  ACT-320,  ACT-330, 
AOS-250  and  support  contractor  personnel.  The  personnel  will 
receive  training  in  the  areas  of  multitasking  operating  systems 
and  communication  protocols  and  networks,  as  applicable.  This 
training  will  be  provided  by  each  organization.  Training  for  the 
ITWS  specific  operations  will  include  software,  hardware,  system 
operator,  configuration  management,  and  test  tool  training.  This 
training  will  be  funded  by  the  program  office,  conducted  by  the 
contractor  on  the  contractor  procured  software  and  will  be 
further  delineated  in  the  SOW.  Training  for  the  test  team  will 
be  completed  prior  to  the  commencement  of  OT&E. 

Additionally,  on  the  job  training  for  the  OT&E  test  team  will  be 
acquired  through  discussions  with  the  ITWS  contractor  throughout 
the  development  and  DT&E  process,  reading,  and  studying  various 
system  information  contained  in  manuals  and  other  system 
documentation.  Experience  will  also  be  gained  through  hands-on 
system  operation  during  pre-OT&E  activities.  The  projected  ITWS 
test  and  evaluation  personnel  requirements  are  outlined  in  table 
4.5. 1-1. 
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TABLE  4.5. 1-1. 


ITWS  T&E  PERSONNEL  REQUIREMENTS 


ITWS  T&E  Personnel  Requirements 

FY 

96 

97 

98 

99 

00 

01 

Test 

Phase 

DT&E 

DT&E/ 

OT&E 

DT&E/ 

OT&E 

DT&E/ 

OT&E 

OT&E/ 

PAT&E 

PAT&E 

Organizations* 

Total 

MY/Org 

ACT  * 

7.00 

8.00 

11.00 

9.00 

7.00 

5.00 

47.00 

AOS 

2.00 

4.00 

5.00 

5.00 

5.00 

5.00 

26.00 

AT 

1.00 

1.00 

2.50 

1.50 

1.50 

1.00 

8.50 

AF 

0.00 

0.50 

2.00 

3.00 

3.00 

3.00 

11.50 

AND 

2.00 

3.00 

4.00 

4.00 

3.00 

2.00 

18.00 

Prime 

Contr 

actor 

0.00 

3.00 

5.00 

5.00 

2.00 

2.00 

17.00 

Total 

MY/FY 

& 

Test 

Phase 

12.00 

19.50 

29.50 

27.50 

21.50 

18.00 

128.00 

*  Manpower  requirements  include  both  FAA  and  contract  support 
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4»5.2  Test  Articles. 


The  first  delivery  test  articles  will  consist  of  four  complete 
IOC  ITWS  system  suites =  The  first  site  will  be  the  FAA  Technical 
Center,  Atlantic  City  International  Airport,  NJ .  The  second  and 
third  sites  will  be  located  at  the  simple  and  complex  sites  which 
are  presently  TBD,  The  fourth  unit  will  be  delivered  to  the 
Program  Support  Facility  (PSF)  in  Oklahoma  City,  OKo  The  PSF 
unit  will  be  used  for  maintenance  training. 

4.5.3  Test  Sites. 

The  selected  simple  and  complex  site  will  offer  a  broad  spectrum 
of  convective  activity  including  both  air  mass  differentials  and 
frontal  systems.  The  simple  site  will  assess  the  minimum 
interface  functionality,  while  the  complex  site  will  assess  the 
interfaces  with  multiple  airports  and  radar  systems  within  an 
operational  environment.  Both  sites  will  assess  the  interface 
functionality,  operational  effectiveness,  and  suitability, 
including  AF  functions,  within  an  air  traffic  environment. 

Testing  will  be  conducted  at  the  first  three  facilities  listed  in 
section  4.5.2. 

4.5.4  Test  Support  Equipment . 

The  following  is  a  list  of  anticipated  test  equipment  required 
for  the  ITWS  0T6E  test  program.  Additional  test  equipment  may  be 
required  as  the  T&E  program  progresses. 

a.  A  contractor-developed  test  tool  will  be  required  to 
play  back  weather  data  recorded  from  ITWS  input 
sensors.  This  tool  will  be  a  development  contractor 
deliverable  and  will  be  used  at  the  FAA  Technical 
Center  to  develop  and  execute  weather  scenarios  that 
test  the  ITWS  interfaces  and  system  performance. 

b.  Protocol  analyzers  will  be  used  throughout  the  ITWS 
testing  cycle  for  validation,  interpretation,  and 
troubleshooting  of  the  system's  communications. 

c.  MIT/LL  facilities  will  assist  in  offline  Data  Reduction 
and  Analysis  (DR&A)  of  recorded  sensor  data  sets. 

d.  A  test  suite  at  the  FAA  Technical  Center  will  be  required 
to  support  meteorological  data  analysis  of  the  ITWS 
products  and  to  display  raw  input  data  and  intermediate 
products  for  comparison  to  ITWS  products.  The  test  suite 
will  consist  of  several  workstations  displaying  raw 
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sensor  input  and  intermediate  products.  Application 
software  to  support  the  test  suite  will  be  provided  by 
MIT/LL. 

e.  Communication  lines  and  modems  to  support  integration 
testing  at  the  Technical  Center.  Seven  dedicated 
communication  lines  and  modems  are  anticipated  for  the 
interface  testing.  The  lines  will  require  a  minimum  of  a 
9600  KBPS''  and  T-1  rates. 

f.  AOS  support  equipment  will  be  provided  when  available. 

4.6  TEST  CONFIGURATION  MANAGEMENT. 

The  contractor  will  follow  configuration  management  guidelines  in 
accordance  with  the  FAA-STD-021,  FAA-STD-026,  and  FAA  Order 
1800. 8F  for  software  development.  The  configuration  management 
program  will  establish  a  Software  Configuration  Control  Board 
(SCCB).  The  SCCB  establishes  a  baseline  for  software 
requirements,  software  design  and  developed  software.  Any 
changes  to  the  baselined  requirements  or  design  are  submitted  to 
the  SCCB  for  approval.  Requirement  and  design  changes  are 
submitted  via  an  Engineering  Change  Proposal  (ECP). 

As  part  of  the  configuration  management  program,  an  FCA  and  a  PCA 
will  be  conducted.  Configuration  audits  will  be  conducted  by  the 
Program  Office  or  designated  representatives. 

The  FCA  is  a  formal  examination  of  functional  characteristic  test 
data  for  a  configuration  item,  prior  to  acceptance,  to  validate 
that  the  item  has  achieved  the  performance  and  functional 
characteristics  specified  in  its  functional  or  allocated 
configuration  identification.  The  PCA  is  a  formal  examination  of 
the  "as-built"  configuration  of  a  unit  to  verify  that  it  conforms 
to  its  technical  design  in  order  to  establish  the  configuration 
item's  initial  product  configuration  identification. 

During  testing,  the  test  team  will  be  responsible  for  documenting 
the  configuration  during  each  phase  of  testing.  Test  procedures 
and  data  sheets  will  be  used  to  document  any  configuration 
changes.  Test  data  will  be  analyzed  in  accordance  with  any 
configuration  changes. 

4.7  TEST  PLANNING  WORK  GROUP. 

The  TPWG  consists  of  representatives  from  ACT-320,  ASD-140,  ATQ- 
3,  ATR-330,  ASD-120,  AOS-250,  and  AND-460.  The  TPWG  will  meet 
routinely  to  discuss  test  related  issues  including  the  TEMP  and 
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test  plan  and  procedure  development,  DEMVALs,  OT&E  schedules, 
etc.  The  organizational  representatives  will  provide  input  for 
test  requirements  and  represent  their  respective  organizations 
regarding  acceptance  of  test  responsibilities.  MIT/LL  and  the 
(development  contractor  are  ad  hoc  members  and  will  be  included  as 
required. 

4.8  METEOROLOGICAL  EVALUATION  PANEL. 

A  panel  of  independent  experts  from  various  scientific  and 
technical  organizations  (ACT-320,  AOS-250,  NSSL)  will  perform  an 
assessment  of  ITWS  product  performance.  The  panel  will  be 
ongoing  and  will  provide  input  to  test  personnel  and  assist  in 
resolving  DT&E  and  OT&E  issues. 

5.  T&E  PROGRAM  DESCRIPTION. 

5.1  COMPLETED  DT&E/PATSE. 

There  has  been  no  DT&E/PAT&E  performed  on  the  ITWS.  The  DT&E 
test  phase  will  begin  at  completion  of  the  TRR. 


5.2  COMPLETED  OT&E . 

All  completed  OT6E  was  conducted  during  the  demonstration  phase. 
An  informal  DEMVAL  was  conducted  at  Orlando  and  Dallas  in  1993. 
This  DEMVAL  was  an  initial  verification  of  the  ITWS'  capability 
to  meet  AT  weather  requirements  and  produce  scientifically  valid 
products.  An  independent  science  panel  was  convened  in  1993  to 
review  the  science  of  the  ITWS  algorithms.  Based  on  the  results 
of  the  1993  DEMVAL  and  Science  Panel  findings,  the  program  office 
decided  to  proceed  with  a  formal  DEMVAL  in  1994.  The 
Demonstration  Phase  OT&E  (DEMVAL)  was  conducted  at  Orlando  and 
Memphis  during  the  summer  of  1994.  The  test  reports  are  listed 
below. 

DOT/FAA/CT-TN95/1  ITWS  Operational  Test  and  Evaluation 

(OT&E)  Demonstration/Validation  at 
Orlando,  FL  and  Dallas  Fort-Worth,  TX 
(May-September  1993),  DOT/FAA/CT-TN95/ 1 . 


ACT-320  Report 


ITWS  Demonstration/Validation  Phase  OT&E 
Final  Report,  (Draft),  February  1995. 
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The  following  are  the  ITWS  COIs  and  status  from  the  1994  DEMVAL. 


a.  Are  the  ITWS  products  useful  during  operationally  significant 
weather  in  terms  of  their  availability,  timeliness,  and 
suitability  for  AT  use?  Are  detections  and  false  alarm  rates 
acceptable  to  users? 

Status ;  Partially  resolved  during  the  demonstration  phase; 
further  product  suitability,  detection  and  false  alarm 
evaluations  required. 

b.  Are  the  ITWS  products  displayed  without  the  need  for  further 
meteorological  interpretation?  Is  the  displayed  information 
understandable? 

Status:  Partially  resolved  for  the  demonstration  phase;  further 
ongoing  verification  will  be  conducted  during  the  development 
phase . 

c.  Does  the  ITWS  reduce  (perceived)  controller  workload  during 
adverse  weather  conditions  in  the  terminal  area? 


Status :  Partially  resolved  for  the  demonstration  phase;  further 
ongoing  verification  will  be  conducted  during  the  development 
phase . 

d.  Do  ITWS  products  enhance  the  effectiveness  of  traffic 
planning/management  during  adverse  weather  conditions  in  the 
terminal  area?  Are  terminal  airspace  and  runways  planned  for  and 
used  more  effectively? 

Status:  Partially  resolved  for  the  demonstration  phase;  further 
ongoing  verification  will  be  conducted  during  the  demonstration 
phase . 

e.  Does  the  unavailability  of  interfacing  systems/sensors 
adversely  affect  ITWS  operations? 

Status:  Partially  resolved  during  the  demonstration  phase;  the 
unavailability  of  certain  input  sensors  (e.g.,  TDWR,  LLWAS) 
during  the  DEMVAL  did  not  preclude  the  ITWS  from  producing  useful 
products  to  AT  personnel.  Since  during  DEMVALs,  the  interface 
products  were  distributed  by  MIT/LL  (not  IRD  requirements). 

These  results  do  not  carry  over  for  the  development  system. 
However,  the  DEMVAL  results  do  provide  a  framework  for  further 
assessing  this  COI.  Therefore,  during  OT&E  the  system  reaction 
to  sensor  loss  and  subsequent  reacquisition  of  sensor  input  will 
be  evaluated  along  with  the  user  response  to  sensor  loss. 
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f.  Will  input  sensor  quality  be  adequate? 


Status ;  Partially  resolved  for  the  demonstration  phase.  ITWS_ 
produced  accurate  products  for  the  available  input  sensors  during 
the  1994  DEMVAL. 

5.3  DEMONSTRATION  TEST  &  EVALUATION  (DT&E)  TESTING. 

5.3.1  DT&E  Program  Description. 

DT&E  is  conducted  to  assist  in  the  engineering  and  development 
process  by  determining  the  degree  to  which  functional  engineering 
specifications  are  addressed.  DT&E  includes  test  and  evaluation 
of  subsystem  hardware  and  software  on  full-scale  engineering 
models . 

DT&E  will  be  conducted  in  accordance  with  FAA-STD-1810 . 4B  and  the 
FAA  approved  CMTP.  ACT-320  will  be  responsible  for  monitoring 
all  aspects  of  the  ITWS  DT&E  testing  cycle. 

DT&E  will  verify  that  the  specification  requirements  of  FAA-E- 
ITWS  are  met  and  that  the  developed  interfaces  are  correctly 
implemented  according  to  applicable  IRDs .  Elements  of  the  ITWS 
DT&E  phase  are  depicted  in  figure  5.3. 1-1.  DT&E  for  the  ITWS 
will  consist  of  DT&E  DQT,  DT&E  FQT,  DT&E  FAT,  and  DT&E  SAT. 

These  DT&E  phases  are  discussed  in  the  following  sections. 

5. 3. 1.1  DT&E  Design  Qualification  Tests  (DT&E  DQT) . 

DT&E  DQT  tests  will  be  conducted,  comprising  of  informal  software 
unit  testing  and  software/hardware  integration  testing  on 
components  and  subsystems  during  the  development  process .  As 
part  of  DQT,  software  tests  will  determine  the  accuracy  of  the 
software  developed  to  implement  the  government^furnished 
meteorological  and  display  algorithms. 

5.3. 1.2  DT&E  Functional  Qualification  Tests  (DT&E  FQT). 

DT&E  FQT  tests  will  be  performed  on  the  computer  software 
configuration  item,  computer  software  component  and  computer 
software  unit  requirements  identified  in  the  software 
requirements  specification  and  associated  interface  documents. 

As  part  of  FQT,  software  tests  will  determine  the  accuracy  of  the 
software  developed  to  implement  the  government-furnished 
meteorological  and  display  algorithms  functions  on  the  ITWS 
platform  in  accordance  with  system  requirements.  The  tests  will 
verify  that  the  developed  software  is  consistent  with  the 


algorithms  and  the  software  requirements  specification 
requirements . 
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FIGURE  5.3. 1-1 .  DT&E  ELEMENTS 
46 
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5. 3. 1.3.1  DT&E  Reliability. 

The  ITWS  contractor  will  conduct  a  reliability  program  in 
accordance  with  FAA-E-ITWS  and  the  ITWS  SOW.  The  test  procedures 
will  conform  to  the  technical  requirements  of  MIL-STD-785B,  Tasks 
101-104,  201-204,  and  303.  Each  system  will  meet  a  minimum  Mean 
Time  Between  Failures  (MTBF)  of  2704  hours.  MTBF  data  will  be 
collected  throughout  DT&E  and  OT&E  testing.  Reliability  will  be 
demonstrated  during  DT&E  and  as  reguired  during  OT&E  Integration, 
OT&E  Operational,  OT&E  Shakedown,  PAT&E,  and  Field  Shakedown 
testing.  ACT-320  and  AOS-250  will  verify  the  ITWS  meets  this 
reliability  reguirement.  Contractor  testing  during  DT&E  FAT  will 
include  an  operability  test  to  ensure  the  ITWS  can  operate 
without  failure  for  an  extended  period  of  time. 

5. 3. 1.3. 2  DT&E  Maintainability. 

The  ITWS  contractor  will  conduct  a  maintainability  program  in 
accordance  with  FAA-E-ITWS  and  the  ITWS  SOW.  The  program  will  be 
carried  out  according  to  the  reguirements  of  MIL-STD-470B,  Tasks 
101-104,  201-203,  205,  207,  301  and  302.  The  Mean  Time  To  Repair 
(MTTR)  will  not  exceed  0.5  hour.  A  maximum  time  allowed  for  a 
single  repair  will  be  in  accordance  with  the  reguirements  of  FAA- 
E-ITWS  and  the  SOW.  Maintainability  will  be  demonstrated  during 
DT&E  and  as  reguired  during  OT&E  Integration,  OT&E  Operational, 
OT&E  Shakedown,  PAT&E,  and  Field  Shakedown  testing.  ACT-320  and 
AOS-250  will  verify  this  maintainability  reguirement. 

5. 3. 1.3. 3  DT&E  Availability. 

The  ITWS  will  be  tested  to  verify  that  the  system  meets  the 
availability  reguirements  from  the  FAA-E-ITWS  and  SOW.  The 
availability  of  the  system  interfaces  will  also  be  examined  to 
determine  the  availability  of  the  sensor  inputs.  The  contractor 
will  demonstrate  and  verify  operational  availability  of  .9998. 

For  the  purposes  of  operational  availability  calculations,  the 
MTTR  is  taken  as  the  total  time  of  all  interruptions  of  service 
regardless  of  the  cause  or  duration  of  each.  Service 
interruptions  do  not  include  natural  disasters.  Availability 
will  be  demonstrated  during  DT&E  and  as  reguired  during  OT&E 
Integration,  OT&E  Operational,  OT&E  Shakedown,  PAT&E,  and  Field 
Shakedown  testing.  ACT-320  and  AOS-250  will  verify  this 
availability  reguirement. 
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5.3. 1.4  DT&E  Factory  Acceptance  Testing  (FAT). 


As  part  of  the  DT&E  effort,  the  contractor  will  plan  and  conduct 
a  DT&E  FAT  of  the  ITWS,  which  will  be  formally  witnessed  by  the 
FAA.  The  DT&E  FAT  will  verify  the  capability  of  ITWS  to  meet  all 
functional,  interface,  and  performance  requirements  of  the  ITWS 
Specification  FAA-E-ITWS.  DT&E  FAT  is  conducted  at  the 
contractor's  facility  and  will  be  conducted  by  the  contractor 
using  FAA-approved  contractor  test  plans  and  procedures.  The 
contractor  is  responsible  for  the  timely  and  satisfactory 
completion  of  testing  in  accordance  with  the  ITWS  schedule. 


A  72-hour  system  operability  test  will  be  conducted  as  part  of 
FAT.  The  test  will  utilize  a  representative  sample  of  test 
cases,  including; 

a.  archiving  and  playback  of  scenarios; 

b.  switching  SDs  to  TDWR  and  LLWAS,  then  back  to  SD  mode; 
and 

c.  switching  the  system  from  product  generation  mode  to 
maintenance  mode  for  a  period  of  no  less  than  4  hours 
and  then  switching  back  to  the  product  generation  mode, 

5.3. 1.5  DT&E  Site  Acceptance  Testing  (SAT). 

DT&E  SAT  is  performed  by  the  system  developer  on  the  first 
article(s)  system  delivered  to  each  of  the  test  sites  before 
acceptance  of  the  subsystem  by  the  FAA  CAI .  The  purpose  of  this 
testing  is  to  ensure  that  the  systems  are  properly  installed, 
ready  for  operation,  and  include  all  necessary  equipment.  This 
testing  is  monitored  by  government  personnel. 

As  part  of  DT&E  SAT,  Delta  DT&E  will  be  conducted  by  the 
contractor  at  the  FAA  Technical  Center.  This  will  test  the 
interfaces  which  were  not  available  at  the  contractor's  facility 
and  exist  within  the  FAA  Technical  Center.  Additionally,  RMS 
functionality  will  also  be  tested.  DT&E  SAT  will  be  conducted  in 
accordance  with  the  FAA-approved  contractor  DT&E  SAT  test  plan, 

5.3.2  Development  Contractor  Test  Documentation. 

The  development  contractor  will  prepare  a  CMTP  in  accordance  with 
the  Data  Item  Description  (DID)  and  Contract  Data  Requirements 
List  (CDRL)  specified  in  the  SOW.  This  CMTP  provides  information 
that  pertains  to  the  contractor  conducted  testing  in  DT&E  and 
PAT&E.  This  document  will  show  traceability  to  the  ITWS 
specification  and  VRTM.  Additionally,  the  contractor  will 
develop  and  have  FAA  approval  of  the  DT&E  plan.  These 
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plans /procedures  will  have  pass/fail  criteria  for  each  of  the 
requirements  tested. 


The  CMTP  will  be  developed  from  the  DT&E  and  PAT&E  test 
requirements  stated  in  the  Quality  Assurance  section  of  the  FAA- 
E-ITWS  and  the  SOW.  The  CMTP  will  be  updated  as  required 
throughout  the  contract  to  reflect  any  ECPs  and/or  contract 
modifications  that  alter  the  testing  program.  DT&E  test  plans 
will  be  prepared  in  accordance  with  the  DIDs  and  CDRLs  specified 
in  the  SOW. 

5.3.3  Government  Responsibilities. 

The  government  will  prepare  the  ITWS  Specification  (FAA-E-ITWS) , 
SOW  and  TEMP  documents.  The  TEMP  will  contain  the  VRTM  and  the 
VRTM  will  contain  the  ITWS  requirements  specified  in  NAS-SS-1000, 
Volumes  I-V.  Presently,  the  ITWS  specification  is  not  baselined. 
When  baselined,  the  VRTM  will  be  updated  to  map  the  specification 
requirements  to  the  NAS  requirements.  The  revised  VRTM  will  be 
submitted  for  TPRC  approval. 

This  VRTM  will  be  used  to  ensure  that  the  prime  contractor  tests 
the  same  set  of  requirements  upon  which  the  system  was  designed. 
The  government  will  identify  CPPs,  COIs  and  Exit  Criteria  (see 
sections  3.4.1,  3.5,  and  3.4.2)  that  will  be  used  to  transition 
the  ITWS  from  DT&E  to  OT&E  and  OT&E  to  KDP-4.  OCD  will  not  be 
performed  on  the  ITWS.  CPPs,  COIs,  and  exit  criteria  are  being 
added  to  the  ' Notes /Remarks '  column  of  the  VRTM  where 
appropriate . 

5.4  PRODUCTION  ACCEPTANCE  TEST  &  EVALUATION  TESTING  (PAT&E). 

The  contractor  will  conduct  PAT&E  on  each  production  unit  to 
verify  that  the  product  conforms  with  all  provisions  of  the 
contract  and  meets  the  stated  requirements.  The  tests  will  be 
conducted  in  accordance  with  the  FAA-approved  PAT&E  test  plans 
and  procedures.  The  Technical  On-site  Representative  (TOR)  will 
also  be  involved  with  this  test  phase  to  assist  in  maintaining 
the  facility  data  reference  file  and  to  ensure  user  readiness  for 
system  sell  off.  Operational  elements  of  the  PAT&E  phase  are 
depicted  in  figure  5.4-1.  The  contractor  will  conduct  testing  of 
the  ITWS  at  each  operational  facility  before  acceptance  and 
deployment  of  the  subsystem  by  the  FAA  CAI . 

5.4.1  PAT&E  Factory  Acceptance  Testing  (PAT&E  FAT). 

During  PAT&E,  FAT  is  conducted  by  the  contractor  at  their  factory 
for  each  delivered  item  to  verify  that  it  conforms  to  applicable 
specifications  and  requirements.  A  limited  subset  of  DT&E  FAT 


requirements  will  be  performed  on  each  PAT&E  article «  Successful 
completion  of  PAT&E  FAT  represents  a  partial  FAA  acceptance  of 
the  production  article. 

5.4.2  PAT&E  Site  Acceptance  Testing  (PAT&E  SAT). 

PAT&E  SAT  testing  is  conducted  to  verify  the  effective 
installation  of  the  ITWS  into  each  facility  and  to  verify  the 
suitability  of  the  system  for  Field  Shakedown  testing  and  AT 
operations.  The  TOR  will  also  be  involved  with  this  test  phase 
to  assist  in  maintaining  the  facility  data  reference  file  and  to 
insure  user  readiness  for  system  sell  off. 
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PRODUCTION  ACCEPTANCE  TEST  &  EVALUATION 


FIGURE  5.4-1.  PAT&E  ELEMENTS 
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5.5  OT&E  TESTING. 


The  OT&E  testing  will  be  performed  in  the  following  three  phases; 

Phase  1;  Interface/Integration  Facility;  (FAA  Technical 

Center,  Atlantic  City  International  Airport,  NJ) ; 

Phase  2;  Operational  Site;  simple  site  (TBD); 

Phase  3;  Operational  Site;  complex  site  (TBD). 

This  phased  approach  ensures  a  structured  methodology  for 
examining  and  resolving  COIs  and  OT&E  requirements  (see 
subsections  5.5. 1.1.1  "  5.5. 1.1.3  and  5.5.2  -  5. 5. 2. 2). 
Additionally,  it  provides  a  framework  that  requires  verification 
that  the  system  has  met  the  exit  criteria  and  can  proceed  into 
the  next  phase.  Elements  of  OT&E  are  depicted  in  figure  5.5-1. 

OT&E  Shakedown  will  be  conducted  independently  of  OT&E 
Operational  and  Integration  testing.  Requirements  tested  during 
the  OT&E  Operational  and  Integration  testing  and  determined 
acceptable,  will  not  be  retested  during  OT&E  Shakedown.  These 
requirements  will  be  addressed  in  the  OT&E  Shakedown  test  plan. 

The  OT&E  Integration  test  environment  will  include  the  FAA 
Technical  Center  for  initial  interface  integration  testing, 
followed  by  testing  in  AT  operational  environments.  ACT-320  will 
develop  the  OT&E  Test  Plans  and  Procedures  in  accordance  with  FAA 
Order  1810. 4B  and  FAA-STD-024b.  These  Plans  and  Procedures  will 
present  each  requirement  with  pass/fail  criteria. 


AT  and  AF  personnel  will  be  actively  involved  in  the  hands-on 
evaluation  of  the  ITWS  equipment.  The  OT&E  will  be  conducted 
primarily  in  an  operational  environment  to  evaluate  the 
effectiveness  and  suitability  of  the  ITWS  into  the  NAS. 

The  planned  approach  for  resolution  of  COIs  during  OT&E  is 
presented  in  section  3.5.  Test  limitations  that  may  impact  the 
resolution  of  the  COIs  and  impede  the  ITWS  OT&E  testing  are  ITWS 
dependencies  on  other  systems  for  input  data  and  the  absence  of 
hazardous  weather  phenomena.  The  CPPs  found  in  section  3.4.1 
will  be  tested  in  accordance  with  the  VRTM  found  in  section  7  of 
this  document. 
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OPERATIONAL  TEST  &  EVALUATION 


FIGURE  5.5-1.  OT&E  ELEMENTS 


5.5.1  OT&E  Integration  Testing. 


OT&E  Integration  consists  of  testing  the  NAS  system  end-to-end 
performance.  The  ITWS  VRTM  lists  the  requirements  from  the  NAS- 
SS-1000,  Volumes  I-V,  that  will  be  tested  throughout  the  test 
phases . 

5.5. 1.1  OT&E  Integration  Test  Program. 

This  testing  will  ensure  the  successful  integration  of  NAS 
systems,  subsystems,  and  end-to-end  performance  requirements. 

The  subsystems  will  be  tested  in  a  NAS  system  equivalent 
environment.  If  an  interface  is  not  available,  then  a  simulation 
will  be  utilized.  The  ultimate  goal  of  integration  testing  is  to 
ensure  that  the  new  system’s  end-to-end  performance  does  not 
adversely  impact  operational  NAS  systems  or  subsystems  and  that 
system  performance  achieves  the  goals. 


The  OT&E  Integration  will  employ  a  structured  methodology  to 
verify  the  interoperability  of  the  ITWS  into  the  NAS.  A  building 
block  approach  that  first  verifies  the  communications  layers  in 
accordance  with  approved  communication  models  will  be  used.  Once 
testing  is  completed,  applications  processes  for  data  transferred 
will  be  conducted.  This  testing  will  initially  be  conducted  on 
each  individual  interface,  followed  by  testing  on  multiple 
interfaces , 
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5.5. 1.1.1  Phase  1  -  OT&E  Integration  Testing. 

The  purpose  of  the  interface/integration  facility  testing  is  to 
mitigate  interface  problems  before  they  are  encountered  in  an 
operational  environment.  This  testing  will  utilize  the 
interfaces  available  within  the  FAA  Technical  Center.  Limited 
operational  testing  will  also  be  conducted  at  this  facility  by 
using  real~time  data  acguired  from  operational  sensors  located  at 
the  Atlantic  City  International  (ACY)  and  Philadelphia 
International  (PHL)  airport  facilities.  The  impact  of  sensor 
input  loss  to  the  ITWS  and  to  ATC  personnel  will  be  initially 
assessed.  RMS  integration  testing  will  also  be  addressed  during 
this  phase.  Refer  to  the  Phase  I  configuration  diagram,  figure 
5.5. 1. 1. 1-1. 

Sensor  Inputs  FAATC  System  User 


FIGURE  5.5. 1.1. 1-1.  FAA  TECHNICAL  CENTER  OT&E  INTEGRATION 

PHASE  I  CONFIGURATION 
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5. 5. 1.1. 2  Phase  2  -  OT&E  Integration  Testing. 

The  OT&E  Integration  testing  will  be  conducted  in  an  AT 
operational  facility.  Simple  site  testing,  currently  TBD,  will 
assess  interface  functionality  within  an  AT  environment.  This 
will  verify  the  ITWS  capability  of  interfacing  with  all  designed 
system  inputs.  This  configuration  will  include  a  single  input 
from  each  of  the  ITWS  interfaces.  Refer  to  the  Phase  II 
configuration  diagram,  figure  5.5.1. 1.2-1.  OT&E  operational 
testing  will  also  be  conducted  at  this  facility. 


Sensor  Inputs 


System  Users 


FIGURE  5. 5. 1.1. 2-1. 


SIMPLE  SITE  OT&E  INTEGRATION 
PHASE  II  CONFIGURATION 


5.5. 1.1.3  Phase  3  -  OT&E  Integration  Testing. 

The  complex  site,  currently  TBD,  will  be  conducted  in  an  AT 
operational  facility.  It  is  analogous  to  the  simple  site  testing 
with  the  exception  that  the  complex  site  will  ingest  inputs  from 
multiple  radars  and  airports.  Refer  to  the  Phase  III 
configuration  diagram,  figure  5. 5. 1.1. 3-1.  OT&E  operational 
testing  will  also  be  conducted  at  this  facility. 

Sensor  Inputs  System  Users 


FIGURE  5. 5. 1.1. 3-1.  COMPLEX  SITE  OT&E  INTEGRATION 

PHASE  III  CONFIGURATION 
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5.5. 1.2  Schedule. 


Refer  to  the  Integrated  Schedule  in  appendix  B. 

5. 5. 1.3  Key  Sites;  Anticipated  OT&E  integration  test  sites. 

a.  FAA  Technical  Center,  Atlantic  City  International 
Airport,  NJ; 

b.  simple  site,  TBD; 

c.  complex  site,  TBD. 

5.5. 1.4  Training. 

AF  personnel  participating  in  DT&E/OT&E  will  require  training  on 
the  operations  (hardware  and  software)  of  the  ITWS  prior  to 
utilizing  the  products  and  participating  in  testing.  AF 
personnel  involved  in  testing  will  have  a  level  of  competence 
with  respect  to  training  (e.g.,  training  on  C,  C++,  UNIX,  etc.). 
Training  (e.g.,  hardware  operation  or  software)  will  be  developed 
by  the  contractor  on  the  contractor  developed  software  in 
accordance  with  the  terms  of  the  SOW.  The  AT  personnel  will  also 
require  training  on  the  ITWS  in  preparation  for  the  1995  and  1996 
testing.  Additional  training  requirements  are  defined  in  section 
4.5.1  of  this  document. 

5.5. 1.5  Personnel. 

The  following  personnel  will  be  required  to  prepare  for  and 
conduct  testing; 

a.  ACT-320  will  provide  test  engineers,  meteorologists, 
support,  and  Human  Factors  personnel  to  develop  plans, 
procedures  and  reports,  and  to  conduct  OT&E  testing. 
Personnel  requirements  and  estimated  costs  are  presented  in 
paragraph  4.3.  Additionally,  ACT-320  will  require  planning 
support  to  assist  the  Program  Office  in  maintaining  a 
workable  schedule  and  will  track  action  items  and  record 
proceedings  of  the  Test  Schedule  Status  Reviews  (TSSR). 

b.  ACT-330  will  provide  test  engineers  to  draft  plans, 
procedures,  conduct  testing,  and  draft  reports  for  RMS 
testing.  Budgeting,  staffing,  and  training  test  personnel 
will  be  the  responsibility  of  ACT-330. 

c.  AOS-250  personnel  requirements  are  defined  in  section 
5. 5. 3. 3. 
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d.  AT  personnel  will  provide  trained  operators  to  participate 
in  the  operational  test  environment  to  evaluate  the 
effectiveness  and  suitability  of  the  ITWS. 

e.  AF  regional  technicians  will  be  required  for  system  and 
maintenance  support.  A  Memorandum  of  Agreement  (MOA)  will 
be  prepared  and  coordinated. 

f.  Prime  contractor  support  for  OT&E  is  defined  within  the  SOW 
and  will  occur  prior  to  software  development  completion. 

The  following  list  prime  contractor  OT&E  support: 

1.  The  contractor  support  for  OT&E,  will  at  a  minimum, 
ensure  the  communication  software  provides 
connectivity  to  live  interfaces. 

2.  The  contractor  must,  for  a  minimum  6-month  period, 
provide  the  following  services  to  support  the 
government  conduct  of  OT&E  testing:  systems 
engineering,  hardware  engineering,  software 
training,  communication  hardware  and  software,  and 
hardware  maintenance  personnel  at  the  FAA  technical 
center,  "simple"  site  and  "complex"  sites. 

3.  The  contractor  will  provide  the  factory  software 
development  facility  and  support  services. 

5.5.2  OT&E  Operational  Testing. 

The  following  sections  describe  the  OT&E  Operational  Test  Program 
that  will  verify  the  operational  effectiveness  and  suitability  of 
ITWS  to  fulfill  its  NAS  mission  from  both  an  AT  and  AF 
perspective.  Key  sites,  personnel  training,  and  test 
methodologies  are  similar  to  those  described  in  the  OT&E 
Integration  activities  described  in  section  5.5.1.  Operational 
elements  of  OT&E  are  depicted  in  figure  5.5-1.  Each  of  the 
elements  comprising  the  OT&E  Integration  testing  is  defined  in 
subsections  a  through  j . 

a.  Reliability. 

FAA  Technical  Center  personnel  will  collect  failure  data  which 
will  be  scored  as  a  Relevant  Failure  or  a  Nonrelevant  Failure 
(not  inherent  to  the  equipment).  ACT-320  will  review 
reliability  data  provided  during  DT&E  to  perform  a  reliability 
trend  analysis.  At  that  time,  ACT-320  will  recommend  to  the 
program  office  if  additional  reliability  testing  is  required 
during  OT&E. 
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The  trend  analysis  will  be  used  by  ACT-320  to  predict  system 
reliability  to  the  configuration  item  level.  Failure  rate 
analysis  will  be  based  on  actual  operational  experience,  test 
data,  experience  with  similar  systems,  and  manufacturer 
specifications . 

b.  Availability. 

Failure  data  and  maintainability  test  results  will  be  used  in 
computing  the  ITWS  system  availability  throughout  all  testing 
cycles.  This  analysis  will  be  presented  in  the  final  DT&E 
Report. 

c.  Degraded  Operations. 

Degraded  operations  will  be  addressed  by  making  the  system 
operational  with  maximum  data/sensor  inputs.  The  maximum  will 
be  determined  from  the  ITWS  Radar/LLWAS  Usage  List.  Each 
sensor  will  be  disabled  to  determine  the  algorithm  capabilities 
to  produce  a  result  and  for  the  system  to  adjust  to  the 
abnormal  operation  from  a  data/sensor  input  failure.  The 
transition  to  the  TDWR  and  LLWAS  modes  will  be  observed.  The 
transition  to  the  ITWS  mode  will  be  observed  as  the  data/sensor 
input  is  returned  to  available.  Additionally,  the  NAS  will  be 
monitored  to  ensure  that  the  degraded  operations  do  not 
adversely  impact  other  NAS  systems  (e.g.,  NADIN  II,  ADAS, 
etc . ) . 

d.  Stress  and  NAS  Loading. 

The  system  will  be  tested  for  stress  and  loading  by 
implementing  maximum  data/sensor  inputs  beyond  the  complex  site 
requirements.  The  maximum  data/sensor  input  will  be  determined 
from  the  ITWS  radar/LLWAS  Usage  List.  The  systems  response  to 
the  maximum  data/sensor  input  will  demonstrate  the  systems 
capabilities  and  verify  the  performance.  The  simulation  will 
be  accomplished  through  the  use  of  the  test  data  injector  and 
live  interfaces. 

e.  Human  Factors. 

Human  Factors  personnel  will  assess  end  users  performance  in 
conducting  tasks  and  human  engineering  issues  associated  with 
the  system.  Human  factors  personnel  will  evaluate  the 
operations  of  the  ITWS  in  order  to  determine  if  there  are  any 
adverse  effects  on  air  traffic  controller  efficiency, 
productivity,  and  safety.  Additionally,  the  effect  of  the  ITWS 
on  AT  planning/management  and  perceived  controller  workload 
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will  be  assessed.  These  evaluations  will  take  place  at  each 
operational  test  site  using  live  and/or  canned  data. 


f.  Safety. 


The  ITWS  safety  program  will  be  in  accordance  with  MIL-STD-882C 
and  tasks  101/  102/  104/  106/  202/  204/  205/  302/  and  303  in 
the  ITWS  SOW.  Testing  will  include  physical  safety  (electrical 
shock)  and  safety  from  a  Human  Factors  viewpoint  (i.e./ 
obstruction/  ease  of  repair/  etc.). 

g.  Maintainability. 

The  ITWS  will  be  tested  by  ACT-320  to  predict  system 
maintainability  to  the  configuration  item  level.  System 
fault (s)  will  be  generated  during  the  OT&E  Operational  test 
cycle  to  verify  contractor  maintenance  reguirements .  These 
reguirements  include  the  ability  of  maintainers  to  diagnose  and 
isolate  faults/  repair  or  replace  modules/  and  perform 
operational  checkout  of  the  repaired  item  in  accordance  with 
the  ITWS  maintenance  concept  and  procedures. 

Certification  of  the  procedures  to  determine  the  ITWS  readiness 
to  be  put  back  into  operation  after  repair  will  be  demonstrated 
by  the  contractor  and  reviewed  by  the  FAA.  Maintenance 
documentation/  including  technical  manuals  and  maintenance  log 
entries  will  be  reviewed/observed  for  clarity  and  detail.  The 
latest  baselined  software  version  will  be  reinstalled  on  the 
system  after  an  introduced  system  fault  is  corrected. 

h.  Site-Adaptation  Data. 

OT&E  will  verify  that  the  ITWS  is  adaptable  to  function 
successfully  in  various  site  configurations.  System 
performance  should  not  be  adversely  impacted  by  the  altering  of 
system  parameters  to  meet  various  site  configurations.  This 
adaptable  parameters  will  be  verified  throughout  OT&E  testing 
at  the  sites. 

i.  Security. 

The  ITWS  security  program  will  be  incorporated  into  the  site 
implementation  plan.  Testing  will  include  physical  security 
reguirements  where  applicable  and  operator  security  level 
(i.e./  system  administration/  supervisor  privileges/  etc.). 

The  ITWS  will  have  the  added  physical  security  of  the 
environment  in  which  it  resides. 


1.  Transition  Switchover. 

The  ITWS  will  be  evaluated  to  determine  the  effects  its  removal 
has  on  degradation  of  the  NAS  environment.  This  includes  the 
availability  and  operability  of  backup  systems  for  the  ITWS 
when  the  system  is  not  operational.  The  NAS  environment  will 
be  monitored  for  adverse  impacts  of  system  loss . 

5.5.2. 1  Operational  Test  Site  Locations. 

ITWS  OT&E  test  sites  will  be  selected  in  different  regions  of  the 
United  States  to  offer  a  range  of  weather  phenomena.  This  will 
ensure  that  the  ITWS  is  evaluated  in  various  weather  data 
processing  capacities.  Additionally,  the  sites  selected  consist 
of  both  simple  and  multiple  sensor  configurations.  These 
configurations  are  delineated  by  area  of  coverage  for  single  or 
multiple  airports  with  co-located  ARTCCs .  The  simple  site 
configuration  will  provide  single  radar  sensor  inputs  while  a 
complex  site  configuration  will  have  multiple  radar  inputs. 

Refer  to  section  5.5. 1.1  for  site  configurations. 

There  will  be  Four  First  Article  Systems.  Of  these,  OT&E 
Integration  testing  will  be  conducted  on  three  units,  which 
include  a  unit  at  the  FAA  Technical  Center  for  interface  and 
Integration  testing,  and  an  operational  simple  and  complex  test 
site  (locations  to  be  determined  (TBD)).  A  simple  site  is 
defined  as  a  single  airport  TRACON  with  single  TDWR,  Airport 
Surveillance  Radar  Model  9  (ASR-9),  and  Next  Generation  Weather 
Radar  (NEXRAD)  inputs.  A  complex  site  is  defined  as  a  multi - 
airport  TRACON  with  multiple  TDWR,  ASR-9,  and  possibly,  NEXRAD 
inputs.  The  fourth  unit  will  be  delivered  to  the  PSF  in  Oklahoma 
City  for  maintenance  procedural  and  shakedown  test  development. 

5. 5. 2. 2  OT&E  Operational  Risks  That  May  Adversely  Impact  Test 
Completion. 

a.  The  ITWS  capabilities  need  to  be  tested  with  controlled 
input  data  in  order  to  vary  the  loading  conditions  and  minimize 
testing  time.  It  is  planned  to  have  this  data  provided  to  ITWS 
via  a  test  tool.  This  test  tool  will  utilize  weather  data 
recorded  from  around  the  country  over  a  prolonged  period  of 
time.  Even  with  convective  weather  conditions,  this  weather 
data  may  be  insufficient  to  provide  the  input  data  intensity 
required  to  test  the  ITWS  past  its  critical  performance 
thresholds , 

b.  Although  the  schedule  allows  time  to  collect  weather  data, 
due  to  the  inability  to  control  weather  phenomena,  sufficient 
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data  may  not  be  collected  within  the  time  allotted.  Therefore, 
testing  delays  may  be  incurred. 

c.  Delays  in  the  TDWR  installation  schedule  may  adversely 
impact  the  ITWS  OT&E  schedule. 

5. 5. 2. 3  Schedule. 

See  integrated  schedule  in  appendix  B. 

5.5.3  OT&E  Shakedown. 

OT&E  Shakedown  testing  will  determine  the  overall  readiness  of 
the  ITWS  through  the  exercising  of  the  system  in  an  operational 
environment  to  support  determination  that  the  system  is  ready  for 
full  operation  as  part  of  the  NAS.  This  will  include  testing  to 
confirm  that  when  the  ITWS  is  operated  and  maintained  by 
operational  personnel  in  an  operational  environment,  all 
requirements  are  met  (see  figure  5.5-1).  Shakedown  testing  will 
verify  the  effectiveness,  suitability,  maintainability, 
supportability,  and  integration  requirements  of  the  system. 

Effectiveness  will  assess  the  system's  ability  to  provide 
reliable  service  for  consistent  product  delivery  under  multiple 
operational  conditions.  Suitability  will  assess  the  system 
products  and  user/system  transactions.  Maintainability  will 
assess  the  ease  of  maintaining  the  system  throughout  the 
operational/nonoperational  states  and  to  assess  the  system 
capabilities  of  displaying  system  statuses  to  the  operator. 
Supportability  will  ensure  that  the  system's  hardware  and 
software  is  capable  of  being  maintained  with  the  given 
documentation  and  training. 

System  integration  testing  will  assess  the  ITWS  operation  and 
determine  any  negative  impact  on  the  existing  air  traffic 
control.  Testing  will  include  an  assessment  of  the  external 
inputs  to  the  ITWS  which  include  the  systems  as  shown  in  figure 
3.2-1. 

5.5.3. 1  OT&E  Shakedown  Organizations. 

Shakedown  testing  will  be  conducted  by  AOS-250  at  the  simple  and 
complex  sites.  The  actual  performance  of  the  Shakedown  tests 
will  be  by  AF  personnel  and  regional  maintenance  technicians  who 
will  use  the  services  provided  by  the  ITWS  and  who  will  have  the 
maintenance  responsibility  for  the  selected  test  sites. 
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5. 5. 3. 2  OT&E  Shakedown  Preparation. 


anc;-250  will  develop  the  OT&E  Shakedown  Test  Plan  and  Procedures 
accorSice  with  FAA  Order  1810. 4B  and  FAA-STD-024B. 

r.^.3.3  ht&K  Shakedown  Personnel  and  Training. 

The  following  personnel  will  be  required  to  prepare  for  and 

conduct  OT&E  Shakedown  testing. 

fl  AOS-250  hardware  and  software  engineers  will  develop  the 
“ws  sLkSdown  Test  Plan  and  Procedures,  prepare  reports, 

and  conduct  testing «. 

b.  Regional  technicians  wlll^be  requlred^for^system^and^^^^^ 

^Ssl^dlleclorwlll  coordinate  this  effort  with  the  region. 

c.  AOS-250  meteorologists  will 

meteorological  performance  of  the  ITWS  algorithms. 

d.  Evaluation  of  training  and  maintenance  will  be  performed  as 
part  of  OT6E  Shakedown. 

5. 5. 3. 4  Training. 

AT  personnel  participating  in  the  Shakedown  Resting  will  require 

training  on  thenlTWS^prlor^to  utilising  the^Products^^^^_^^^^^^^  ^ 

aS’^wIll^’as  AOS-250  engineers,  participating  Jn  Sh^edown^testing 

risier^eiieel^'anS  sSiSriesIIig  ^Both  areas  of  training 
"S  L  deilloped  and  conducted  by  the  contractor  in  accordance 

With  the  ITWS  SOW. 

5. 5. 3. 5  OT&E  Shakedown  Test  Implementation. 

Shakedown  testing  will  be  conducted  in  two  phases: 

Phase  1 :  Simple  Operational  Site 

Phase  2%  Complex  Operational  Site 

The  simple  site  configuration  will  provide  single  radar  sensor 
inoulrlsee  figure  5. 5. 1.1. 2-1)  while  a  complex  site 
Configuration  will  have  multiple  radar  inputs  (see  figure 

5. 5. 1.1. 3-1). 
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Shakedown  testing  at  the  simple  site  configuration  will  include 
all  elements  of  validation  and  verification  as  shown  in  figure 
5.5-1.  Shakedown  testing  at  the  complex  site  configuration  will 
be  centered  around  validating  and  verifying  system  integration  o 
the  multiple  radar  inputs. 

Delays  in  the  TDWR  and  NEXRAD  installation  schedules  may 
adversely  impact  the  ITWS  Shakedown  test  schedule. 

6.  VERIFICATION  REQUIREMENTS  TRACEABILITY  MATRIX. 

Presently,  the  ITWS  specification  is  not  baselined.  When 
baselined,  the  VRTM  will  be  updated  to  map  the  specification 
requirements  to  the  NAS  requirements.  The  revised  VRTM  will  be 
submitted  for  TPRC  approval.  The  current  VRTM  is  presented  in 

appendix  A. 


7.  INDEPENDENT  OPERATIONAL  TEST  AND  EVALUATION. 

Independent  OT&E  will  be  conducted  on  the  ITWS  by  the  ATS  Test 
Team.  Details  will  be  provided  at  a  later  date. 


8.  ACRONYMS  AND  ABBREVIATIONS- 


ACF 

ADAS 

AF 

AGL 

AIV 

ALDAR.S 

AP 

APME 

APMQ 

APMT 

ASOS 

ASR 

ATC 

ATCT 

ATIS 

ATR 

AWOS 

AWP 

CAB 

CAI 

CDR 

CDRL 

CMTP 

CPP 

COI 

COTS 

DCE 

DEMVAL 

DFW 

DID 

DLP 

DQT 

DRR 


Area  Control  Facility 

ASOS/AWOS  Data  Acquisition  System 

Airway  Facilities 

Above  Ground  Level 

Aviation  Impact  Variable 

Automated  Lightning  Detection  and  Reporting  System 
Anomalous  Propagation 

Associate  Program  Manager  for  Engineering 

Associate  Program  Manager  for  Quality 

Associate  Program  Manager  for  Test 

Automated  Surface  Observing  System 

Airport  Surveillance  Radar 

Air  Traffic  Control 

Air  Traffic  Control  Tower 

Automated  Terminal  Information  System 

Air  Traffic  Plans  &  Requirements  Service 

Automated  Weather  Observing  System 

Aviation  Weather  Products 

Configuration  Control  Board 

Contract  Acceptance  Inspection 

Critical  Design  Review 

Contract  Data  Requirements  List 

Contractor's  Master  Test  Plan 

Critical  Performance  Parameters 

Critical  Operational  Issues 

Commercial-Of f-The-Shelf 
Data  Communications  Equipment 

Demonstration/ Validation 

Dallas-Fort  Worth  International  Airport 
Data  Item  Description 
Data  Link  Processor 
Design  Qualification  Test 
Deployment  Readiness  Review 
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DTE 

DT&E 

EXCOM 

FAA 

FAATC 

FAATG 

FAT 

FCA 

FSD 

GFE 

GFI 

ICfiiA 

ICD 

ICDDT 

IMCS 

IOC 

lOT&E 

IRD 

ISO 

IPT 

ITWS 

KDP 

LLWAS 

MCF 

MCO 

MDCRS 

MEM 

MIT/LL 

MAOPR 

MNS 

MOA 


Data  Terminating  Equipment 
Development  Test  and  Evaluation 
Executive  Committee 
Federal  Aviation  Administration 

Federal  Aviation  Administration  Technical  Center 
Federal  Aviation  Administration  Telecommunications 
Gateway 

Factory  Acceptance  Testing 
Functional  Configuration  Audit 
Full  Scale  Development 
Government  Furnished  Equipment 
Government  Furnished  Information 
Initial  Checkout  and  Acceptance 
Interface  Control  Documents 

Interface  Control  Document  Database  Development 
Tool 

Interim  Monitor  Control  Software 
Initial  Operational  Capability 

Independent  Operational  Testing  and  Evaluation 
Interface  Requirement  Documents 
International  Standards  Organization 
Integrated  Product  Team 
Integrated  Terminal  Weather  System 
Key  Decision  Point 

Low  Level  Wind  Shear  Alert  System 
Metroplex  Control  Facility 
Orlando  International  Airport 

Meteorological  Data  Collection  and  Reporting 
System 

Memphis  International  Airport 

Massachusetts  Institute  of  Technology  Lincoln 
Laboratory 

Minimum  Acceptable  Operational  Requirements 
Mission  Need  Statement 
Memorandum  of  Agreement 
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MPS 


MTBF 

MTTR 

WAS 

WCP 

NEXRAD 

WSSL 

NWS 

WWSTG 

OCD 

ORD 

ORD 

OS  I 

OT&E 

PAT&E 

PCA 

PD 

PDR 

PFA 

PM  . 

POD 

PIP 

PSF 

PSN 

PUP 

QRO 

RBDT 

RPG 

RMA 

RMMS 

RMS 

RUC 

SAT 


Maintenance  Processor  Subsystem 
Mean  Time  Between  Failure 
Mean  Time  To  Repair 
National  Airspace  System 
NAS  Change  Proposal 
Next  Generation  Weather  Radar 
fijational  Severe  Storms  Laboratory 
National  Weather  Service 

National  Weather  Service  Telecommunications 
Gateway 

Operational  Capability  Demonstration 

Operational  Requirements  Document 

Operational  Readiness  Demonstration 

Open  System  Interconnection 

Operational  Test  and  Evaluation 

Production  Acceptance  Test  and  Evaluation 

Physical  Configuration  Audit 

Program  Directive 

Preliminary  Design  Review 

Probability  of  False  Alarm 

Program  Manager 

Probability  of  Detection 

Program  Implementation  Plan 

Program  Support  Facility 

Packet  Switching  Network 

Principle  User  Processor 

Quality  Reliability  Officer 

Ribbon  Display  Terminal 

Radar  Product  Generator 

Reliability,  Maintainability  and  Availability 

Remote  Maintenance  Monitoring  System 

Remote  Monitoring  Subsystem 

Rapid  Update  Cycle 

Site  Acceptance  Testing 


SAV 

SD 

SOW 

STD 

STP 

T&E 

TBD 

TCCC 

TDWR 

TEMP 

TOR 

TPRC 

TPWG 

TRACON 

TRR 

TSSR 

VRTM 

WMO 


State  of  Atmosphere  Variable 
Situation  Display 
Statement  of  Work 
Standard 

Software  Test  Plan 
Test  and  Evaluation 
To  Be  Determined 
Tower  Control  Computer  Complex 
Terminal  Doppler  Weather  Radar 
Test  and  Evaluation  Master  Plan 
Technical  On-site  Representative 
Test  Policy  Review  Committee 
Test  Planning  Work  Group 

Terminal  Radar  Approach  Control  facility 

Test  Readiness  Review 

Test  Schedule  Status  Reviews 

Verification  Requirements  Traceability  Matrix 
World  Meteorological  Organization 


69 


APPENDIX  A 
VRTM 


(A 

m  -O 


© 


<  = 

11  a 
<  a 
...  re 
c 

o  o 

^  z 

II 

is  X 
(A  .. 
C  t- 

o  ^ 
c  >» 
c  re 
o  ^ 

•j?! 

« 

©  .o 

*ir  ^ 

^1 

B  iT 

I- 

c  o 

.s  ^ 

ts  ^ 
re  0) 
o  a 

S  « 
■c  c 


o  ^ 

O  o 

T  O 
CO  II 
CO  X 

©  II 

s?. 

re 

£  -C 
O)  © 


O  X 
>  HI 
^  II 

g^g 
0^0 
T  S:  CO 

CO  o  il 

GQ  S  O 
.  II  © 
—  HI  a 
o  --w 
>>< 
g^ili 

2  o  d 
CO  o  □. 

OT-TW 

Jf  ">  « 
^  w  o 

®  A  - 

8  o  < 

i;  i-  II 
O  ©  ^ 

C  -J  D. 
g  JCO 
c  o  > 
g>  «J  re 

*5  s=  5* 

O'—  O 
©On 

a:>^ 


Reaulrement  Source-  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level):  D=SS-1000  Vol  V;  E=MAOPR:  F=Exit  Criteria;  G=CPP;  H=COI,  ^Inspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  WI=SOW 


w 

H  a 
<  a 
..  n 
c  ^ 
o  o 
'Zi  z 

s  >< 
<0  .. 
c 

o  <1> 
c  >* 

C  flj 
®  — 

“I 
“I 
«  >» 
d)  £ 

V  *0 

»  o 


l> 

£  II 
0)  J 

li 

.5  ® 

(Q  O) 

a 
it  w 
^  c 


o  ^ 

O  o 

V  O 
w  » 

CO  X 
6^" 

0)  II 


BB 

X  L. 

r'o 

” 

O  X 

>  UJ 

§“■5 

2  K  o 

^  Si 

CO  o  1 

*(?  <  s 

m  s  o' 

.  li  d> 
—  UJ  Q. 

o^“«9 

>  ^  < 
00” 

§>-i 

^  o  u 
*00 
CO  O  Q. 
«p  'T-  CO 

<  CO  0 
O  “ 

8  Q  < 
^  ir  II 
□ 

00. 

4.J  9  O 

c  -I  a 

g  5  CO 

c  o  >, 
£  -I 


to 

< 

0) 

D) 

re 

r 

</) 

^  6  Jl 
m  s  o 

.  ii  Q> 

—  uj  a 


CO  O  D. 
CO  ^  CO 

^  W  o 

"of 

Q  O  << 

t;  » 
O  "fl)  ^ 
w>  >  O 

♦-  ®  <l) 
C  “*  Q. 
®  ^  CO 

c  o  >« 
®  _J  ra 

5=  “■ 

S  >  A 


C/) 

© 

fl 

<  .= 

H  a 
<  a 
..  (0 
c  ^ 
o  o 

«  II 
W  .. 

c  »- 
o  e> 
c  >> 
c  © 
©  — 
o 

M  I 

°  i 

iT 

Cfl 

©  ^ 

II 

1  = 
§1 
«  g 

Q- 
c  c 

5:iE 


o  O 
\  O 
w  H 
(f)  s 

-.i:' 

sru 

©  II 
>  tD 

-J  to 
-c  *c: 

X  'C 
-  O 


A-8 


A- 9 


o  X 
>  ^ 

§^5 

®  a:  o 

^  CL  w 

(On" 

‘(?|  = 
CD  2  u 
^  II  0) 

—  uj  a 

I  >2 


CO  O  Q. 
OT  ^  (0 

ii*  ^ 

<  CO  o 

;.;  II  5* 


3  ^  v; 

O  0)  ^ 

^  C)  0) 
C  -I  Q. 
©  5  CO 

l5 1 

3=  “• 
O’—  D 
©  O  ti 

a:  >  -^ 


«  © 

fl 

<  = 

H  Q. 

<  a 
..  © 
c  ^ 
o  o 

©  II 
X 

« 

c 

o  Q> 
c  5^ 
c  © 
©  — 

°  I 

°  i 

w  >* 
©  ^ 

H* 

£  © 


1| 

©  J 

li 

.S 

©  © 
o  a 
£  « 


o  O 
^  O 
«fl  11. 
CO  ^ 

sro 

©  II 

SO 

j  re 

£  ‘C 
O)  © 


—  «.9 

>  ^ 
«  11 

gu.: 

^  o 

“f 

-  ^ 
o 

>  ^ 

o  o 

I  o 

CO  o 
CO 

Jt  O 

to 


(A 

1 « 
<  = 
II  a 
<  Q. 
..  ro 
c 

o  o 
73  Z 
w  II 
X 
<A  .. 

c  ^ 
o  a> 
c 

C  (Q 
®  — 

?| 

D  g 


(A  >> 
0)  £ 

P 

ij 

m  ^ 
(Q  C) 

O  O. 

E  w 

If 


o  . 

O  o 

T  O 
CO  *1 
(O  X 

-.fc 

-'  « 
£  “ 
O)  Q> 

X  *z 

r'o 


O  X 
>  UJ 
^  II 

§“-5 

2  K  o 
^  Q,  CO 
CO  n  " 

I?!®. 

03  S  u 
.  II  0) 
—  UJ  Q. 
O  --CO 
>>< 
o  o  II, 

§>-!. 

^  o  o 

CO  O  Q. 


<  </)  o 

g  Q  < 
i;  ^  II 
3  SI2'^ 
O  0)  ■*■ 

4^  0) 

c  -I  a 
g  5  CO 

15  S' 
=  =  £• 
cr—  Q 
5  P  II 
a:  > 


«  <D 

<:^ 
H  a 
<  a 
..  ra 

C  4^ 

o  o 

z 

n  II 
is  X 
w  .. 

g  fe 

E  ^ 

<D  — 

Q  g 


*»  “o 

a  0) 

^  5= 


I! 

ii 

.2  ^ 
■ti  ^ 

I  ^ 

it  w 
u  c 

5:il 


o  o 
'T  O 
W  » 
W  X 

-.fc 

trO 

0)  II 
0) 

J  jo 

D) 

*—  4-» 

X  x 

ro 


O  X 

g»h5 
£  afO 
^  CL  w 
«  O  iL 

‘(?i  = 

CD  S  u 
.  II  0) 
—  lU  Q. 

>  >  < 
O  O  H 
§>"1. 
o  o 

■  O  fl) 

(0  o  a 

w  ^  w 

i  CO  ^ 

<  <0  o 

•  •  II  O) 

8q< 

3 

o  O  *• 

4-  9  fl) 
C  -J  o. 
«  »« 

fl)  -J  _« 
3  =:  £• 

®  o 
DO  -n 


A-13 


W  © 


<  = 

H  Q. 

<  a 

..  CO 

c 

o  o 

i=  >< 
*0  .. 
c  t- 
o  0) 
c 

is 

e.i 

w  >» 
o  ^ 

n" 

•’^1 

IE 

ll 

■■s  i 

a 

£  « 
•c  c 


o  O 
*7  O 
CO  » 
(/)  X 

-.fc 

cro 

0)  II 
_l 

£  C 
O)  0) 

X  X 

ro 


«  fl) 


(3 


0) 


< 

n  CL 
<  a 
ro 

c  ^ 

o  o 

«  II 

i:  X 
w 

c  ^ 
o  o 
c  >• 
c  «j 

Q>  “- 

Q 

A  $ 

9.1 

«  >. 

0)  ^ 

M* 

11 

I  i 

1  = 

O  s 

-tt  ^ 
(3  fi> 

O' 
it:  w 
s:  c 


o  O 

^  A 

co  ^ 

-I* 

S'O 

0)  II 

S  o 

— *  « 

£  ‘C 
O)  0) 

••—I  M 

X  u 

r'o 

—  4..» 

o  « 

>  UJ 

<0  o  .y, 

» 1  - 
QQ  ^  u 

.  H  « 
“  iiJ  G 


>> 

o  o 

»-  o 


o 

0) 

a 

CO 


o 

o> 


CO  o 
CO 

<  CO 

8  Q  < 
o  ©  ^ 

<-»  fl) 

c  -J  a 
®  ^  CO 
c  O  >t 

2  .j  ffi 

3  =  ^ 
O’—  Q 

<S  5  ” 

Ct  >  "3 


Shkdwnj  Thresholds  |  Notes/Remarks 


o  ^ 

®  ef  o 

TT  Q.  c/) 

*"  i 

ffl  ^  U 

^  II  0) 

—  wo. 

>  rff 


<  w  o 

■  -  II  CD 

8d< 


€0  >  O 
^  ®  0) 
C  -J  Q. 

®  ^  W 

c  o  >, 

£  -J  ^ 

3  =  £• 

Q 

&  o  n 
c:  > 


A- 18 


<A 

|1 

<  = 

il  °* 
<  a 
..  (0 
C  4^ 

o  o 

z 

n  II 
^  X 
M  .. 

®  t 

E  ^ 

q>  — 

"I 

Q  I 


(A  >« 
Q)  ^ 

h" 

i"  a> 


o| 

5  iT 

0)  ^ 

ii 

.2  44 

CO  0) 

a  a 
E  w 
c  c 

5:ji 


o  O 
T  U 
OT  » 
(0  X 

iro 

»  II 

”*  is 

-O 


O'—  o 

O  O  II 


Requirement  Source:  A=SS-1000  Vol  I.  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test:  D=Demonstration:  A=Analysis: 

Vol  II  (Low  Level):  D=SS-1000  Vol  V;  E=MAOPR:  F=Exlt  Criteria;  G=CPP;  H=COI,  l=lnspection:  L=Verifled  by  lower  layer;  X=Not  applicable 

J=Dlptay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Veriflcation  Method:  T=Test:  D=Demonstration:  A=Analysis: 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Notes/Remarks 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstratlon;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=C08,  ^Inspection;  L=Verified  by  lower  layer;  X=f^ot  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  il  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exlt  Criteria;  G=CPP;  H=COI,  l=lnspectlon;  L=Verlf!ed  by  lower  layer;  X=Not  applicable 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level):  D=SS-1000  Vol  V;  E=MAOPR:  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Reauirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test:  D=Demonstration:  A=Analysis; 

Vol  II  (Low  Level)-  D=SS-1000  Vol  V;  E=MAOPR:  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 


SrcDoc  [  Requirement  Definition  Ops  Shkdwn  j  Thresholds 

1  201  ILLWAS  WINDS;  UPDATE  RATE  []  f  f  \  X  \  D  1 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS“1000  Vol  II  (High  Level);  C=SS-1000  Verification  Rflethod:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E^H/IAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  f=!nspection;  L=Verified  by  lower  layer;  X=Hot  applicable 


Shkdwn  j  Thresholds  Wotes/Remarks 

Lead-In 


Requirement  Source;  A=SS-1000  Vol  I,  B=SS-1000  Vo!  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Oemonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Dlplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Shkdwnl  Thresholds  I  Notes/Remarks 


g  « 

il  Q- 

<  a 
..  re 
c  ^ 
o  o 
5  2 
2  11 

w  .. 

§  I 

“I 

w  >» 
o  n 
^  *0 
i  0) 
^  £ 

P 

£  II 
<u  J 


§5 

’Si  ^ 
re  0) 

O  Q. 

E  « 
S  = 
>  iL 


o  O 
T  O 
CO  H 
(O  X 

£^0 
»  II 

-*  re 
x:  “c 
m  0) 

X  £ 

“**  ^ 

o 

>  UJ 

§“■5 
s  q;  O 
T  £  « 

«  O  H 
'(?<  = 
8 

—  UJ  a 

>  ^  < 
o  o  II 

g  >  “* 

^  o  d 
•  o  0) 
CO  O  Q. 

«•>:<« 
il-  <»  i; 
^  & 

fl)  Jl  - 
o  ^  ^ 
t;  H 

o  <i>  . 

CO  >  O 

g  5  CO 

£  -j  re^ 

3  =  £- 
O’—  D 


srcDoc  [  Requirement  Definition 
1  249  [Detection  Capability 


Reauirement  Source-  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration:  A=Anaiysis; 

Vol  li  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR:  F=Exit  Criteria;  G=CPP;  H=COI.  !=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  R/i=SOW 


c  j—  — 


O 


Q-  TO 


CO  .E 
>  £ 
£-L 


CO 


261  [The  ITWS  shall  generate  a  graphical  product  using  radar  tornado  detections. 


262  TORNADO  MESSAGE/ALERT  I  I  I  D  D  X  |  LEAD-IN 


CO 


O 


O 


w  0, 

n  a 
<  a 
re 
c 

o  o 
^  Z 
2  II 
ti  X 
i/i  .. 

c  ^ 
O  A> 
c 

c  re 

<D  — 

"I 

“  I 


M  "O 

SlT 

0)  _l 

If 

m  ^ 

I  a 

s  «> 

c  c 

5:j[ 


o  . 

o  O 
\  O 

w  41 

CO  X 

0>  II 


-O 
—  ^ 
o  X 

>  UJ 

^  11 

S  d:  o 

T  £  CO 

CO  o  " 

i?i= 

03  oS  u 
.  II  a> 
—  UJ  a 

>  5!  < 

o  o  i> 
o  >  -I 

^  o  o 

CO  O  Q. 
W  Y  CO 
ir  CO 
<  CO  o 

O  « 

s  Q  < 
i;  ^  n 
o  ©  ^ 

♦-  ®  © 
c  -I  a 

g  5  CO 

i5  s- 

3  =  £■ 
O’—  Q 
®  o  II 

J3C  >  “> 


-33 


Shkdwn  Thresholds  [  Notes/Remarks 

1  |l^d-in 


Requirement  Source:  A=SS-1000  Voi  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstratlon;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  S=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  R^=SOW 


A-35 


(A 

<  Q. 

re 

c 

o  o 
■'5  2: 

rt  II 
^  X 
</>  .> 
c  ^ 
o  ^ 
c  >» 

s  iS 
re  — 

“I 
«  >* 
re  £ 

£i 

ii 

7  II 

1^- 
5  o 
o  5= 

to  S 

«  s. 
£  « 
•c  c 

5:  z 


O  o 
v  o 
CO  41 
CO  X 

iro 
re  II 


D)  re 
X  X 
rro 


o  X 
>  ^ 

g“-g 

®  qTO 
’T  £  CO 
CO  o  >■ 

H 

CD  S  u 
.  Ii  re 

—  UJ  Q. 
o  ‘“W 
>>< 
o  o 
§>-1. 
<1-0  0 

CO  O  Q. 
W  -7  CO 

^  CO  O 

re  Jl  - 

o  O  < 

t  ^  II 

o  re  ^ 

"  S  8 

4_l  W  0 

C  -•  Q. 

i 

C  O  >, 

re  -I  ^ 
3  ~  °- 

O"—  Q 
re  o  !i 
X  >  -> 


Thresholds  Notes/Remarks 


Requirement  Source:  A=SS-1000  Vo!  I,  B=SS-1000  Vo!  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstratlon;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=EKlt  Criteria;  G=CPP;  H=COi,  l=lnspection;  L=Verifled  by  lower  layer;  X=Hot  applicable 

J=DiDlav  Spec.  K=Algor  Spec,  L=A-Spec,  M=SOW 
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A-37 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=inspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Dipiay  Spec»  K=Algor  Spec,  L=A-Spec,  M=SOW 


A-39 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Rflethod:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  tl  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=CO!5  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=AlgorSpec,  L=A-Spec,  M=SOW 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level):  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Dlplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 
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A-42 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level):  D=SS-1000  Vol  V;  E=MAOPR:  F=Exit  Criteria;  G=CPP:  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Requirement  Source;  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Anaiysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Dlplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Requirement  Source;  A=SS-1000  Vol  I,  B=SS-1000  Vol  !l  (High  Level);  C=SS-1000  Verification  Method;  T=Test;  D=Demonstration;  A=Analysls; 

Vol  ll  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COl,  NInspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 
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Requirement  Definition  |  Ops  Bnt  Shkdwn  |  Thresholds  [  Notes/Remarks 

Will  receive  and  process  current  surface  weather  observation,  at  least  once  every  T  T  X  1 

minute. 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  il  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=CO!,  ^Inspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=D]play  Spec,  K=AlgorSpec,  L=A-Spec,  M=SOW 


Requirement  Definition 

The  ITWS  shall  accept  national  gridded  weather  data  and  iVleteorological  Data 
Collection  and  Reporting  System  (MDCRS)  products  from  the  National 
Meteorological  Center  (NMC)  which  will  broadcast  via  FAATSAT. 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  ll  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A^Analysis; 

Vol  ll  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exlt  Criteria;  G=CPP;  H=COI,  l-lnspection;  L=Verified  by  lower  layer;  X=Mot  applicable 

J=Dlplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 
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Requirement  Source:  A=SS-1000  Vo!  I,  B=SS-1000  Vo!  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysls: 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  8=lnspection;  L=VerifBed  by  lower  layer;  X=h!ot  applicable 

J=Dip!ay  Spec,  K=AlgorSpec,  L=A-Spec,  M=SOW 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  0=88-1000  Vol  V;  E=MAOPR;  F=Exlt  Criteria;  G=CPP;  H=COI,  l=lnspectlon;  L=Verifled  by  lower  layer;  X=Not  applicable 

J=Dlplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Requirement  Source:  A=SS-1000  Vo!  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysls; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  B=lnspectlon;  L=Verlfled  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 
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Requirement  Source:  A=SS-1000  Vol  I,  8=88-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstratlon;  A=Analysis; 

Vol  II  (Low  Level);  0=88-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verlfled  by  lower  layer;  X=Not  applicable 

J=Dlplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  il  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  !=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 


Requirement  Source: 


Requirement  Definition 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test:  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspectlon;  L=Verified  by  lower  layer;  X=Not  applicable 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstratlon:  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 


A-58 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COB5  S=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=:DipBay  Spec,  K=Aigor  Spec,  L=A-Spec,  M=SOW 


A-59 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  l(  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exlt  Criteria;  G=CPP;  H=COI.  Nlnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


SrcDoc  Requirement  Definition  Ops  fnt  Shkdwn  j  Thresholds  Notes/Remarks 

1  473  fTRACON  RANGE  PRECIPITATION  [1  I  I  IlEAD-IN 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Venfied  by  Sower  layer;  X=Not  applicable 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level):  D=SS-1000  Vol  V;  E=MAOPR:  F=Exit  Criteria;  G=CPP;  H=COI,  ^Inspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


SrcDoc  Requirement  Definition  Ops  Int  Shkdwn  Thresholds  Notes/Remarks 

497  The  RMS  shall  transfer  to  the  MPS  only  the  overall  subsystem  status  for  the  T  T  T  D 

standby  equipment: 


A-63 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  ll  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysls; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=SOW 


SrcDoc  Requirement  Definition  Ops  Int  Shkdwn  Thresholds  |  Notes/Remarks 

513  IrmMS  Subsystem  AMCCWS. Request  (a ve  1  sec  max  3  sec)Acknowledgement  (1  T  T  T  D 

sec  ave  -  max  3  sec)Response  Time  (1  sec  ave  -  max  3  sec) 
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A-65 


Requirement  Source:  A=SS-1000  Vol  i,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  M=S0W 


RMS  shall  transfer  a  maintenance  status  change,  for  a  declared  condition 
once  to  the  MRS; 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test:  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=iyiAOPR;  F=Exit  Criteria;  G=CPP;  H=COI,  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 


A-67 
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SrcDoc  Requirement  Definition  Ops  fnt  Shkdwn  Thresholds  Hotes/Remarks 

537  The  RMS  shall  have  the  capability  to  automatically  dial  a  telephone  number  for  S  X  I  X 

establishing  communication  linkage  vwith  the  MRS; 


A-68 


Requirement  Source:  A=SS-1000  Vol  I,  B=SS-1000  Vol  II  (High  Level);  C=SS-1000  Verification  Method:  T=Test;  D=Demonstration;  A=Analysis; 

Vol  II  (Low  Level);  D=SS-1000  Vol  V;  E=MAOPR;  F=Exit  Criteria;  G=CPP;  H=COL  l=lnspection;  L=Verified  by  lower  layer;  X=Not  applicable 

J=Diplay  Spec,  K=Algor  Spec,  L=A-Spec,  R^=SOW 


APPENDIX  B 
SCHEDULE 


